datajams-lbai / datajams-evictions (public) (License: AGPLv3) (since 2020-09-01) (hash sha1)
Extracting Harris County evictions information for Houston DataJams
Clone URLs: https://rocketgit.com/user/datajams-lbai/datajams-evictions ssh://rocketgit@ssh.rocketgit.com/user/datajams-lbai/datajams-evictions git://git.rocketgit.com/user/datajams-lbai/datajams-evictions
first
List of commits:
Subject Hash Author Date (UTC)
Allow passing in query for Case.sqlAllCases() 346862c0720ef9054426048cc95caa8f6b068868 Luigi Bai 2020-09-07 22:43:09
This version works in MS-SQL. 4e3f33f464173c91c3f4ac4b6acb0235ed2d6720 Luigi Bai 2020-09-07 19:11:54
Add headers. 4da1ccbdb2e3e852050de3460bc9ac9ba061ed7a Luigi Bai 2020-09-07 16:28:52
loadDockets now works to local. 96ba8f7419f63be6612fe1efd3e24d5d30eaf20a Luigi Bai 2020-09-06 20:59:06
Fix for new column names. 5791d77e19585c494e51c6062ab45440e5af0903 Luigi Bai 2020-09-06 20:44:50
Add headers 32cb7264c06b28dc0404b715f24fd2d82b66633b Luigi Bai 2020-09-06 20:44:29
Have a working version of xfer - now transfers data to MS SQL Server from SQLITE3. 428e493bcc5b7f718a741db046d08905f47d0f1c Luigi Bai 2020-09-05 22:11:55
Stupid MS SQL Server assumes case insensitive collation for text/char columns. Who does that? 2700846d8c1233d27664fb74b582293ddb8cb32a Luigi Bai 2020-09-05 21:11:17
Fixed MS schema - 4096 too big for varchar. ca3870f7e244d83e58d74c7f231baacbc19ba8fa Luigi Bai 2020-09-04 17:04:05
Remove interstitial console.log 19481f6692d9f93e75aa173bee9716261ef54db0 Luigi Bai 2020-09-04 12:07:33
Added license/copyright info. Added shim for Promise. Rewrote xferData with await and shim. 3930998d32fb2a19239e63b1b07bf1804c7ea46e Luigi Bai 2020-09-04 12:04:32
Fixed a typo. Ugh. 64718362e042d4a987d3161f999de087edb7bc8a Luigi Bai 2020-09-04 03:59:37
Must have an example of the creds.js file to learn from. 13d0e75c21bf8194c53e148e37cc6f85d8152de3 Luigi Bai 2020-09-04 03:59:16
Transfer local data to cloud based MS SQL-Server database. 8035b6dac5ce05ced62474409ab46ca490a6db4a Luigi Bai 2020-09-04 03:50:32
Learned that package-lock.json should also be committed. 20222232bd2e5388b76d3330e6f695b5aacfe393 Luigi Bai 2020-09-03 11:04:05
Fixed SPDX license expression. c894ba842750ff63035c850095ac93399495f4bc Luigi Bai 2020-09-03 11:01:27
Updated README. Added .gitignore, node/npm package.json, and the schema definitions for the data stores. a5baa30d1360514e217250737abf3a4518ab6a65 Luigi Bai 2020-09-02 17:35:09
Initial commit f67ef5b180d66c95e84ee5c5aa0a893376afa5c4 datajams-lbai 2020-09-02 02:28:52
Commit 346862c0720ef9054426048cc95caa8f6b068868 - Allow passing in query for Case.sqlAllCases()
Author: Luigi Bai
Author date (UTC): 2020-09-07 22:43
Committer name: Luigi Bai
Committer date (UTC): 2020-09-07 22:43
Parent(s): 4e3f33f464173c91c3f4ac4b6acb0235ed2d6720
Signer:
Signing key:
Signing status: N
Tree: 5cedf4150330c6fdf96815b1ca1899176706f250
File Lines added Lines deleted
lib/Case.class.js 8 6
updateStatus.js 5 5
File lib/Case.class.js changed (mode: 100644) (index 6f1b438..e95ef48)
... ... class Case {
204 204 this.URL = url; this.URL = url;
205 205 }; };
206 206
207 static sqlAllCases(database, rowCB, completeCB) {
208 database.each(
209 "SELECT case_URL from cases",
207 static sqlAllCases(opts) {
208 if (! opts.database) { throw "Must pass in database."; }
209
210 opts.database.each(
211 (opts.query) ? opts.query : "SELECT case_URL from cases",
210 212 (err, row) => { (err, row) => {
211 213 if (err) { if (err) {
212 214 console.error("Case.sqlAllCases", err); console.error("Case.sqlAllCases", err);
213 215 } else { } else {
214 rowCB && "function" === typeof rowCB && rowCB(new Case(null, row["case_URL"]));
216 opts.rowCallback && "function" === typeof opts.rowCallback && opts.rowCallback(new Case(null, row["case_URL"]));
215 217 } }
216 218 }, },
217 completeCB
219 opts.completionCallback
218 220 ); );
219 221 }; };
220 222
 
... ... class Case {
318 320 }); });
319 321
320 322 req.on('error', e => { req.on('error', e => {
321 rejFN({ msg: "Load Case from URL HTTP", err: e });
323 rejFN({ msg: "Load Case from URL HTTP", url: this.URL, err: e });
322 324 }); });
323 325
324 326 // Ok GO: // Ok GO:
File updateStatus.js changed (mode: 100644) (index 3baa858..66cb763)
... ... opts.connectCallback = (database) => {
30 30
31 31 // Track all the update promises, then wait until they're done: // Track all the update promises, then wait until they're done:
32 32 let promises = []; let promises = [];
33 Case.sqlAllCases(
34 database,
33 Case.sqlAllCases({
34 database: database,
35 35 // This is the callback used for each case: // This is the callback used for each case:
36 caseObj => {
36 rowCallback: caseObj => {
37 37 promises.push( promises.push(
38 38 // Load the data from the website, then // Load the data from the website, then
39 39 caseObj.loadCaseFromURL() caseObj.loadCaseFromURL()
 
... ... opts.connectCallback = (database) => {
50 50 .catch(e => { console.error("loadCaseFromURL", caseObj, e); }) .catch(e => { console.error("loadCaseFromURL", caseObj, e); })
51 51 ); );
52 52 }, },
53 () => {
53 completionCallback: () => {
54 54 // And this is the callback used when the SELECT is finished: // And this is the callback used when the SELECT is finished:
55 55 Promise.all(promises) Promise.all(promises)
56 56 .finally(() => { .finally(() => {
 
... ... opts.connectCallback = (database) => {
61 61 }) })
62 62 ; ;
63 63 } }
64 );
64 });
65 65 }; };
66 66
67 67 DAO.connect(opts); DAO.connect(opts);
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/datajams-lbai/datajams-evictions

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/datajams-lbai/datajams-evictions

Clone this repository using git:
git clone git://git.rocketgit.com/user/datajams-lbai/datajams-evictions

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main