caubert / CSCI_3410 (public) (License: CC BY 4.0) (since 2018-05-16) (hash sha1)
Material for Database Class.
Clone URLs: https://rocketgit.com/user/caubert/CSCI_3410 ssh://rocketgit@ssh.rocketgit.com/user/caubert/CSCI_3410 git://git.rocketgit.com/user/caubert/CSCI_3410
master
List of commits:
Subject Hash Author Date (UTC)
Edited Sol of 4.28, Pb and Sol of 4.29, checked Pb 4.30, edited Pb 4.31, edited the DB Applications Resources section, checked Pb 5.1, edited Pb 5.2 and Sol 5.2, Pb and Sol of 6.1, Pb 7.1, checked Pb 7.2, edited Sol 7.2, Pb and Sol of Pb 7.3, checked Ex 7.1 and edited its solution, checked Ex 7.2 and edited its solution, edited Ex and Sol of 7.3, Ex and Sol of 7.4 edited Ex 7.5 and checked its solution, edited Ex and Sol of 6.1, Ex and Sol of 6.2, checked Ex 6.3 and edited its solution, checked Ex 5.1 and edited its solution, checked Ex 5.2 and edited its solution, edited Ex and Sol of 5.3, checked Ex and Sol of 5.4, checked Ex 5.5 and edited its solution, and edited Ex and Sol of 5.6. 6580a85df655eddd149bb5531a0615ddfe9f810e Crystal 2020-04-22 03:02:20
Updated list of bugs. 17827a6a5b127fa6449b495c1e07359da4682100 aubert@math.cnrs.fr 2020-04-20 19:24:01
Worked on nosql chapter. 38dfcfffce7c06d0eab7256a7a5be1e2481a8505 aubert@math.cnrs.fr 2020-04-20 17:52:55
ALT in img folder 6e6469575d825f6c7360b88d6b387aee709b643c pveeral@augusta.edu 2020-04-20 15:26:47
Cleaned up bib file. 505b1000bca31e4d833bf7739d02d2b7e727e69b aubert@math.cnrs.fr 2020-04-20 05:03:18
rapid adjustments in contrib beb43332953ec4e2a8e4376a90ce58b1234eadba aubert@math.cnrs.fr 2020-04-20 00:10:44
Updated CONTRIB.md c29d920e1efbc84766a3caafc0db2fcab4220b32 pveeral@augusta.edu 2020-04-19 19:28:45
testing 4ece7ba3d5c5d99361ef5eac92bb0848f2ea5318 pveeral@augusta.edu 2020-04-19 18:27:50
Small edit, correcting maefile. 7baa188be7d322e5288b498afbb7beaa96a9770b aubert@math.cnrs.fr 2020-04-19 07:36:08
Cleaned latex files. 79b68f7b709ddeebc8133f7962fe5aabb3376304 aubert@math.cnrs.fr 2020-04-19 06:22:30
Minor corrections in installation manual. 436cee8616c25ccbed8bc406d988c2b4d28420f8 aubert@math.cnrs.fr 2020-04-19 06:19:04
Minor corrections in installation manual. cb8cdfbd506a1344c81aecda055165cc1ca54ece aubert@math.cnrs.fr 2020-04-19 06:17:52
Working on install manual. 3702c6437ee163eb4a61b4d69cffee8c8a76dc3d aubert@math.cnrs.fr 2020-04-19 06:04:22
Worked on makefiles and example file. 4255d5e85bb684349f7f7798455dd8b3a273254b aubert@math.cnrs.fr 2020-04-19 04:56:53
Re-idented some of the code. 124375e6bed1edb96d1bb4bcec8f111c8a3a1197 aubert@math.cnrs.fr 2020-04-19 03:10:02
Java indentation 2b317a12b7ab52bdca576a1bb46b2a2ce295464f guest 2020-04-18 22:21:04
test 6fefa044794ff1d74a3d2493556c836b3dd97e74 guest 2020-04-18 22:18:44
Java indentation 5b0e0eb38484a8c67517a36a438f148bd5efa740 guest 2020-04-18 22:14:01
Worked on install notes. b46b931ef11e3cb7dfe87c7f91ec9d5c558567e6 aubert@math.cnrs.fr 2020-04-17 05:22:20
Started to integrate installation manual to notes. fd27b7686dd4c9d99163cf7badc720cd4a050221 aubert@math.cnrs.fr 2020-04-17 04:09:11
Commit 6580a85df655eddd149bb5531a0615ddfe9f810e - Edited Sol of 4.28, Pb and Sol of 4.29, checked Pb 4.30, edited Pb 4.31, edited the DB Applications Resources section, checked Pb 5.1, edited Pb 5.2 and Sol 5.2, Pb and Sol of 6.1, Pb 7.1, checked Pb 7.2, edited Sol 7.2, Pb and Sol of Pb 7.3, checked Ex 7.1 and edited its solution, checked Ex 7.2 and edited its solution, edited Ex and Sol of 7.3, Ex and Sol of 7.4 edited Ex 7.5 and checked its solution, edited Ex and Sol of 6.1, Ex and Sol of 6.2, checked Ex 6.3 and edited its solution, checked Ex 5.1 and edited its solution, checked Ex 5.2 and edited its solution, edited Ex and Sol of 5.3, checked Ex and Sol of 5.4, checked Ex 5.5 and edited its solution, and edited Ex and Sol of 5.6.
Author: Crystal
Author date (UTC): 2020-04-22 03:02
Committer name: Crystal
Committer date (UTC): 2020-04-22 03:02
Parent(s): 17827a6a5b127fa6449b495c1e07359da4682100
Signer:
Signing key:
Signing status: N
Tree: b0873217eb5e47f5b7ef07c57cd9fb7e05999eaf
File Lines added Lines deleted
notes/lectures_notes.md 91 90
File notes/lectures_notes.md changed (mode: 100644) (index 29afcce..80a77f0)
... ... The u͟n͟d͟e͟r͟l͟i͟n͟e͟^[For technical reasons, underlined words cannot
216 216 Finally, the `pdf` version of the document uses [Linux Libertine fonts](http://libertine-fonts.org/), the `html` version uses [Futura](https://en.wikipedia.org/wiki/Futura_(typeface)). Finally, the `pdf` version of the document uses [Linux Libertine fonts](http://libertine-fonts.org/), the `html` version uses [Futura](https://en.wikipedia.org/wiki/Futura_(typeface)).
217 217 <!-- <!--
218 218 Actually, not a good formatter… Actually, not a good formatter…
219 The `sql` code is formatted using the [Poor Man's T-SQL Formatter](http://architectshack.com/PoorMansTSqlFormatter.ashx).
219 The `SQL` code is formatted using the [Poor Man's T-SQL Formatter](http://architectshack.com/PoorMansTSqlFormatter.ashx).
220 220 --> -->
221 221
222 222 Those lecture notes were created under an [Affordable Learning Georgia](https://www.affordablelearninggeorgia.org/) [Mini-Grant for Ancillary Materials Creation and Revision](https://www.affordablelearninggeorgia.org/about/r13_grantees) ([Proposal M71](https://affordablelearninggeorgia.org/documents/M71_Augusta_Aubert.pdf)). Those lecture notes were created under an [Affordable Learning Georgia](https://www.affordablelearninggeorgia.org/) [Mini-Grant for Ancillary Materials Creation and Revision](https://www.affordablelearninggeorgia.org/about/r13_grantees) ([Proposal M71](https://affordablelearninggeorgia.org/documents/M71_Augusta_Aubert.pdf)).
 
... ... Problem (TRAIN table and more advanced `SQL` coding) +.#train
3740 3740
3741 3741 @problem:train -- Question -.# @problem:train -- Question -.#
3742 3742
3743 : Modify the `CREATE` statement that creates the `TRAIN` table (lines 1--5), so that `ID` would be declared as the primary key. It's sufficient to only write the line(s) that need to change.
3743 : Modify the `CREATE` statement that creates the `TRAIN` table (lines 1--5), so that `ID` would be declared as the primary key. It is sufficient to only write the line(s) that need to change.
3744 3744
3745 3745 @problem:train -- Question -.# @problem:train -- Question -.#
3746 3746
 
... ... Problem (TRAIN table and more advanced `SQL` coding) +.#train
3748 3748
3749 3749 @problem:train -- Question -.# @problem:train -- Question -.#
3750 3750
3751 : Modify the `CREATE` statement that creates the `ASSIGNED_TO` table (lines 13--18), so that it has two foreign keys: `ConductorId` references the `ID` attribute in `CONDUCTOR` and `TrainId` references the `ID` attribute in `TRAIN`. It's sufficient to only write the line(s) that need to change.
3751 : Modify the `CREATE` statement that creates the `ASSIGNED_TO` table (lines 13--18), so that it has two foreign keys: `ConductorId` references the `ID` attribute in `CONDUCTOR` and `TrainId` references the `ID` attribute in `TRAIN`. It is sufficient to only write the line(s) that need to change.
3752 3752
3753 3753 @problem:train -- Question -.# @problem:train -- Question -.#
3754 3754
 
... ... Problem (From Business Statement to ER Diagram to Relational Model -- A Network
7159 7159 > >
7160 7160 > Furthermore, you want to be able to add the patrons in your database. > Furthermore, you want to be able to add the patrons in your database.
7161 7161 > A patron has a name, a unique library card number, and an email. > A patron has a name, a unique library card number, and an email.
7162 > A patron can reserve (put a hold on) multiple copies of documents for up to two weeks, and can borrow multiple copies of documents for one week if it's a video or a disk, and one month if it's a book.
7162 > A patron can reserve (put a hold on) multiple copies of documents for up to two weeks, and can borrow multiple copies of documents for one week if it is a video or a disk, and one month if it is a book.
7163 7163 > Of course, a copy can be borrowed by only one patron, but it can be put on hold for one patron while being borrowed. > Of course, a copy can be borrowed by only one patron, but it can be put on hold for one patron while being borrowed.
7164 7164
7165 7165 #. Draw the ER diagram for this situation. Remember to add all the constraints on your relations. #. Draw the ER diagram for this situation. Remember to add all the constraints on your relations.
 
... ... Problem (Using MySQL Workbench's reverse engineering) +.#reverseeng
7173 7173 *This problem requires you to have successfully completed @problem:mysqlw and @problem:UMLtoRELDriver.* *This problem requires you to have successfully completed @problem:mysqlw and @problem:UMLtoRELDriver.*
7174 7174
7175 7175 Using the relational database schema you obtained in @problem:UMLtoRELDriver, write the `SQL` implementation of that database. Using the relational database schema you obtained in @problem:UMLtoRELDriver, write the `SQL` implementation of that database.
7176 Then, using MySQL Workbench, use the "Reverse Engineering" function to obtain a EER diagram of your database, and compare it with the UML diagram from @problem:UMLtoRELDriver.
7177 Apart from the difference inherent to the nature of the diagram (i.e., UML Vs EER), how are they the same?
7178 How do they differ?
7176 Then, using MySQL Workbench, use the "Reverse Engineering" function to obtain an EER diagram of your database and compare it with the UML diagram from @problem:UMLtoRELDriver.
7177 Apart from the difference inherent to the nature of the diagram (i.e., UML vs EER), how else are they different?
7178 How do they the same?
7179 7179 Is the automated tool as efficient and accurate as you are? Is the automated tool as efficient and accurate as you are?
7180 7180
7181 7181 --- ---
 
... ... Problem (From business statements to dependencies -- KEYBOARD) +.#BusinessToDepe
7188 7188
7189 7189 KEYBOARD(Manufacturer, Model, Layout, Retail\_Store, Price) KEYBOARD(Manufacturer, Model, Layout, Retail\_Store, Price)
7190 7190
7191 A tuple in the KEYBOARD relation contains information about a computer keyboard: its manufacturer, its model, its layout (AZERTY, QWERTY, etc.), the place where it is sold, and its price.
7191 A tuple in the KEYBOARD relation contains information about a computer keyboard; its manufacturer, its model, its layout (AZERTY, QWERTY, etc.), the place where it is sold, and its price.
7192 7192
7193 #. Write each of the following business statement as a functional dependency:
7193 #. Write each of the following business statements as a functional dependency:
7194 7194
7195 #. A model has a fixed layout.
7196 #. A retail store cannot have two different models produced by the same manufacturer.
7195 - A model has a fixed layout.
7196 - A retail store cannot have two different models produced by the same manufacturer.
7197 7197
7198 7198 #. Based on those statements, what could be a key for this relation? #. Based on those statements, what could be a key for this relation?
7199 7199 #. Assuming all those functional dependencies hold, and taking the primary key you identified at the previous step, what is the degree of normality of this relation? Justify your answer. #. Assuming all those functional dependencies hold, and taking the primary key you identified at the previous step, what is the degree of normality of this relation? Justify your answer.
 
... ... Solution to [%D %n (%T)](#problem:library_network)
7523 7523
7524 7524 Note that: Note that:
7525 7525
7526 1 - We want to represent the fact that a _single_ document can have _multiple_ copies, which suggests that DOCUMENT and COPY are two separate entities.
7526 - We want to represent the fact that a _single_ document can have _multiple_ copies, which suggests that DOCUMENT and COPY are two separate entities.
7527 7527 - COPY could be made into a weak entity, OF being the identifying relation. - COPY could be made into a weak entity, OF being the identifying relation.
7528 7528 - Nothing in the statement _forces_ a relationship between the patron and the library to exist, so, by simplicity, we do not add it. However, adding it would not have been a mistake. - Nothing in the statement _forces_ a relationship between the patron and the library to exist, so, by simplicity, we do not add it. However, adding it would not have been a mistake.
7529 7529 - The fact that a COPY has to be of a particular kind does not force the kind attribute to be multi-valued or composite: it just means that if we were representing the domains as well, this attribute would have a particular domain that restricts the values to three possibilities (book, video or disk). - The fact that a COPY has to be of a particular kind does not force the kind attribute to be multi-valued or composite: it just means that if we were representing the domains as well, this attribute would have a particular domain that restricts the values to three possibilities (book, video or disk).
 
... ... Note that:
7554 7554 Solution to [%D %n (%T)](#problem:reverseeng) Solution to [%D %n (%T)](#problem:reverseeng)
7555 7555 ~ ~
7556 7556
7557 We give first the code, then the drawing.
7557 We give the code first, then the drawing:
7558 7558
7559 7559 ```{.sqlmysql .numberLines include=code/sql/HW_Person.sql} ```{.sqlmysql .numberLines include=code/sql/HW_Person.sql}
7560 7560 ``` ```
 
... ... Solution to [%D %n (%T)](#problem:reverseeng)
7566 7566 ## Resources {-} ## Resources {-}
7567 7567
7568 7568 - <http://spots.augusta.edu/caubert/teaching/general/java/> - <http://spots.augusta.edu/caubert/teaching/general/java/>
7569 - If you experience troubles, <https://www.ntu.edu.sg/home/ehchua/programming/howto/ErrorMessages.html#JDBCErrors> might be a good read.
7570 - [@Textbook6, 13.3.2] or [@Textbook7, Chapter 10] is a condensed, but good read.
7571 - Many textbook on Java includes a part on Databases, cf. for instance [@Gaddis2014, Chapter 16].
7569 - If you experience any trouble, <https://www.ntu.edu.sg/home/ehchua/programming/howto/ErrorMessages.html#JDBCErrors> might be a good read.
7570 - [@Textbook6, 13.3.2] or [@Textbook7, Chapter 10] is a condensed, but good, read.
7571 - Many textbooks on Java include a part on Databases, just like this one: [F@Gaddis2014, Chapter 16].
7572 7572
7573 7573 ## Overview ## Overview
7574 7574
 
... ... For a quick introduction to Java, cf. <http://spots.augusta.edu/caubert/teaching
7624 7624 We will write and compile a simple java program that manipulates a simple database^[This program ows a lot to the one presented at <http://www.ntu.edu.sg/home/ehchua/programming/java/jdbc_basic.html>.]. We will write and compile a simple java program that manipulates a simple database^[This program ows a lot to the one presented at <http://www.ntu.edu.sg/home/ehchua/programming/java/jdbc_basic.html>.].
7625 7625 Even if the creation and population of the database could have been done from within the program, we will do it as a preliminary step, using the C.L.I., to make our program simpler. Even if the creation and population of the database could have been done from within the program, we will do it as a preliminary step, using the C.L.I., to make our program simpler.
7626 7626
7627 ### The Database (`sql`)
7627 ### The Database (`SQL`)
7628 7628
7629 7629 For this program, we will use the following database: For this program, we will use the following database:
7630 7630
 
... ... and the program needs to load the driver (which is specific to DBMS) at executio
7679 7679 Of course, if the second step failed, then the program needs to exit gracefully, or to provide debugging information to the user. Of course, if the second step failed, then the program needs to exit gracefully, or to provide debugging information to the user.
7680 7680 The program we will obtain can (normally) be compiled, using something like `javac FirstProg.java`{.bash} (or an equivalent command for windows). The program we will obtain can (normally) be compiled, using something like `javac FirstProg.java`{.bash} (or an equivalent command for windows).
7681 7681 But another refinment is needed when you want to execute it. But another refinment is needed when you want to execute it.
7682 We need to set up the *driver* (or *connector*) to make the java `sql` API and MySQL communicate. To do so,
7682 We need to set up the *driver* (or *connector*) to make the java `SQL` API and MySQL communicate. To do so,
7683 7683
7684 7684 - Go to <https://dev.mysql.com/downloads/connector/j/> - Go to <https://dev.mysql.com/downloads/connector/j/>
7685 7685 - Select "Platform Independent", - Select "Platform Independent",
 
... ... Exercise +.#
8033 8033
8034 8034 Exercise +.# Exercise +.#
8035 8035
8036 : Name three classes in the sql API of java.
8036 : Name three classes in the SQL API of java.
8037 8037
8038 8038 Exercise +.# Exercise +.#
8039 8039
 
... ... Exercise +.#
8046 8046
8047 8047 Exercise +.# Exercise +.#
8048 8048
8049 : Briefly explain what the `next()` method from the `ResultSet` class does, and give its return type.
8049 : Briefly explain what the `next()` method from the `ResultSet` class does and give its return type.
8050 8050
8051 8051 Exercise +.# Exercise +.#
8052 8052
 
... ... Exercise +.#
8120 8120
8121 8121 Solution +.# Solution +.#
8122 8122
8123 : API + driver
8123 : API's and driver's to implement them.
8124 8124
8125 8125 Solution +.# Solution +.#
8126 8126
8127 : Because the program will interact with the environment: if this interraction fails (typically, if the connection does not succeed), then we want to be able to catch the exception and recover from that failure.
8127 : It is important to put the statements that create the connection to the database inside the `try...catch`{.java} statement because the program will interact with the environment if this interraction fails (typically, if the connection does not succeed), for which we want to be able to catch the exception and recover from that failure.
8128 8128
8129 8129 Solution +.# Solution +.#
8130 8130
8131 : You can find them listed at <https://docs.oracle.com/javase/7/docs/api/java/sql/package-summary.html>. We used `Connection`, `DatabaseMetaData`, `ResultSetMetaData`, `PreparedStatement`, `Statement`, …
8131 : You can find them listed at <https://docs.oracle.com/javase/7/docs/api/java/sql/package-summary.html>. We have used `Connection`, `DatabaseMetaData`, `ResultSetMetaData`, `PreparedStatement`, and `Statement` to name a few.
8132 8132
8133 8133 Solution +.# Solution +.#
8134 8134
 
... ... Solution +.#
8136 8136
8137 8137 Solution +.# Solution +.#
8138 8138
8139 : A `Statement` object is used to create a `ResultSet` object, by calling e.g. the `executeQuery` method.
8139 : A `Statement` object is used to create a `ResultSet` object, e.g. by calling the `executeQuery` method.
8140 8140
8141 8141 Solution +.# Solution +.#
8142 8142
8143 : It checks if there is data to read, and if there is, it moves the cursor to read it.
8144 It returns a Boolean.
8143 : The `next()` method checks if there is data to read and, if there is, it moves the cursor to read it.
8144 Its return type is a Boolean.
8145 8145
8146 8146 Solution +.# Solution +.#
8147 8147
 
... ... Problem (A GUEST Java Program) +.#Guest_Java
8266 8266 ```{.java .numberLines include=code/java/GuestProgram.java} ```{.java .numberLines include=code/java/GuestProgram.java}
8267 8267 ``` ```
8268 8268
8269 In the following three exercises, you will add some code where the comment `// INSERT HERE Solution to exercises 1, 2 and 3.`{.java} is to obtain a behavior like the following one (you do not have to reproduce it exactly!), where the user input is underlined, and hitting "enter" is represented by $↵$:
8269 In the following three exercises, you will add some code below the comment `// INSERT HERE Solution to exercises 1, 2 and 3.`{.java} in order to obtain a behavior like the following one (you do not have to reproduce it exactly!).
8270 The user input is underlined, and hitting "enter" is represented by $↵$:
8270 8271
8271 8272
8272 8273 ```{text} ```{text}
 
... ... Problem (A GUEST Java Program) +.#Guest_Java
8280 8281 Oh no, (at least) one of the guest from the black list confirmed their presence! Oh no, (at least) one of the guest from the black list confirmed their presence!
8281 8282 The name of the first one is Marcus Hells. The name of the first one is Marcus Hells.
8282 8283
8283 Do you want to remove all the guests that are on the black list and confirmed
8284 Do you want to remove all the guests that are on the black list and who have confirmed
8284 8285 their presence? Enter "Y" for yes, anything else for no. their presence? Enter "Y" for yes, anything else for no.
8285 8286 ``` ```
8286 8287
8287 8288 You should suppose that `BLACKLIST` contains more than one name, and that some other operations are performed where ……………⌛…………… is (typically, some guests will confirm their presence). You should suppose that `BLACKLIST` contains more than one name, and that some other operations are performed where ……………⌛…………… is (typically, some guests will confirm their presence).
8288 Using batch processing or prepared statements will be a plus, but is not mandatory to solve those exercises.
8289 Using batch processing or prepared statements will be a plus, but is not mandatory to solve these exercises.
8289 8290
8290 8291 #. Write a snippet that #. Write a snippet that
8291 #. Ask the user how many guests they have,
8292 #. For each guest, ask their name (using `key.nextLine()`{.java}, that returns the `String`{.java} entered by the user),
8293 #. For each guest name entered, insert in the `GUEST` table an integer that is incremented after each insertion, the name entered by the user, and `NULL`.
8294 #. Write a snippet such that if there is at least one guest who confirmed their presence and whose name is on the blacklist, a message will be displayed at the screen, containing the name of (at least) one of those guests.
8292 #. Asks the user how many guests they have,
8293 #. For each guest, asks their name (using `key.nextLine()`{.java}, that returns the `String`{.java} entered by the user),
8294 #. For each guest name entered, inserts in the `GUEST` table an integer that is incremented after each insertion, the name entered by the user, and `NULL`.
8295 #. Write a snippet such that if there is at least one guest who confirmed their presence and whose name is on the blacklist, a message will be displayed on the screen containing the name of (at least) one of those guests.
8295 8296 #. Write a snippet that asks the user whenever they want to remove from the guest list all the persons on the blacklist that confirmed their presence, and do so if they enter "yes" (or some variation). #. Write a snippet that asks the user whenever they want to remove from the guest list all the persons on the blacklist that confirmed their presence, and do so if they enter "yes" (or some variation).
8296 8297
8297 8298 ## Solutions to Selected Problems {-} ## Solutions to Selected Problems {-}
 
... ... About the type of attacks, DBMS are exposed to many channels.
8380 8381 Indeed, they can be targeted by Indeed, they can be targeted by
8381 8382
8382 8383 - the "usual" attacks on programs (e.g. buffer overflow), - the "usual" attacks on programs (e.g. buffer overflow),
8383 - the "usual" attacks on on-line services (e.g. denial of service),
8384 - the "usual" attacks on online services (e.g. denial of service),
8384 8385 - the "usual" attacks on systems (e.g. weak authentication, privilege escalation), - the "usual" attacks on systems (e.g. weak authentication, privilege escalation),
8385 8386 - **and** some particular attacks (e.g. `SQL` injections). - **and** some particular attacks (e.g. `SQL` injections).
8386 8387
 
... ... Finally, `code/java/SimpleInjection03.java` shows how to use proper statements t
8495 8496
8496 8497 ### Protections ### Protections
8497 8498
8498 Possible protections from sql injections (-like) includes:
8499 Possible protections from SQL injections (-like) includes:
8499 8500
8500 8501 #. Prepared statements (a.k.a. stored procedures), #. Prepared statements (a.k.a. stored procedures),
8501 8502 #. White list input validation, #. White list input validation,
 
... ... would still leave you exposed, as `table_given_by_user` could mix instructions w
8515 8516
8516 8517 Exercise +.#security1 Exercise +.#security1
8517 8518
8518 : You forgot your password for an on-line service, and click on their "Forgot your password?" link. You enter your email, and receive a few seconds later an email with your original password in it. What is the issue here? What are the next steps you should take?
8519 : You forgot your password for an online service, and click on their "Forgot your password?" link. You enter your email and a few seconds later receive an email with your original password in it. What is the issue here? What are the next steps you should take?
8519 8520
8520 8521 Exercise +.#security2 Exercise +.#security2
8521 8522
8522 : Briefly explain what a prepared statement is, and the benefits it provides.
8523 : Briefly explain what a prepared statement is and the benefits it provides.
8523 8524
8524 8525 Exercise +.#security3 Exercise +.#security3
8525 8526
 
... ... Exercise +.#security3
8530 8531 Solution +.# Solution +.#
8531 8532
8532 8533 : :
8533 The issue is that they are storing your password in clear text, which is an extremely bad practice. This suggests that this service does not care about the security of their users, and that all the information on it should be considered compromised. The next steps are:
8534 The issue is that they are storing your password in clear text, which is an extremely bad security practice. This suggests that this service does not care about the security of their users, and that all the data in it should be considered compromised. The next steps are:
8534 8535
8535 - If the same password was used on different websites, change it immediately,
8536 - Change the password on this website,
8537 - Delete your account on this website, or, if not possible, remove as much information as possible (credit card, address, email, etc.),
8538 - Contact them to express your worries about this flaw,
8539 - (Optional) See if your account has already been hacked, using a service like <https://haveibeenpwned.com/>.
8536 - If the same password was used on different websites, change it immediately.
8537 - Change the password on this website.
8538 - Delete your account on this website, or, if that is not possible, remove as much information as possible (credit card, address, email, etc.).
8539 - Contact them to express your worries about this security flaw.
8540 - (Optional) See if your account has already been hacked using a service like: <https://haveibeenpwned.com/>.
8540 8541
8541 8542
8542 8543 Solution +.# Solution +.#
8543 8544
8544 8545 : :
8545 A prepared statement is stored in a DBMS as a "query with parameters", a template, waiting for values to be passed to fill those placeholders, or slots, and being executed.
8546 It is used to execute the same or similar statements repeatedly with high efficiency:
8547
8548 - Since it is pre-compiled, and compiled only once, it takes less computational resources to be executed.
8549 - In the case where the arguments are transmitted over the network, it means that only the arguments, and not the whole query, has to be sent, which may result in a increase in speed.
8546 A prepared statement is stored in a DBMS as a "query with parameters," or a template waiting for values to be passed to fill those placeholders, or slots, and then is executed all together as one statement.
8547 It is used to execute the same or similar statements repeatedly and with high efficiency, since it is pre-compiled, and compiled only once, it takes less computational resources to be executed. Also, in the case where the arguments are transmitted over the network, it means that only the arguments, and not the whole query, has to be sent, which may result in a increase in speed.
8550 8548
8551 8549 Moreover, since only the arguments are passed, it prevents `SQL` injection, when properly utilized. Moreover, since only the arguments are passed, it prevents `SQL` injection, when properly utilized.
8552 8550
8553 8551 Solution +.# Solution +.#
8554 8552
8555 8553 : :
8556 There are two ways:
8554 There are two ways to test if `SQL` injections are possible:
8557 8555
8558 - Look for places where the program is asking for user-input, and enter values like `1 OR 1 = 1`, or `; DROP TABLE Users;--`
8556 - Look for places where the program is asking for user input and enter values like `1 OR 1 = 1` or `; DROP TABLE Users;--`
8559 8557 - Look for an automated tool (like <http://sqlmap.org/>) that will test the server to which we are connecting. - Look for an automated tool (like <http://sqlmap.org/>) that will test the server to which we are connecting.
8560 8558
8561 Note that both options can be explored in parallel. You can also check e.g. <https://sqa.stackexchange.com/q/1527/> for more ideas on how to test for injections.
8559 Note that both options can be explored in parallel. You can also check out coder resoures, e.g. <https://sqa.stackexchange.com/q/1527/>, for more ideas on how to test for injections.
8562 8560
8563 8561 ## Problems {-} ## Problems {-}
8564 8562
 
... ... Problem (Insecure Java Programming) +.#insecure_java
8570 8568 ```{.java .numberLines dedent=6 include=code/java/InsecureProgram.java snippet=gist} ```{.java .numberLines dedent=6 include=code/java/InsecureProgram.java snippet=gist}
8571 8569 ``` ```
8572 8570
8573 Assume this software is connecting to a schema in a database hosted at <http://example.com/> using
8571 Assume this software is connecting to a schema in a database hosted at <http://example.com/> using:
8574 8572
8575 8573 ```{.java} ```{.java}
8576 8574 Connection conn = DriverManager.getConnection( Connection conn = DriverManager.getConnection(
8577 8575 "jdbc:mysql://example.com/:3306/?user=admin&password=admin"); "jdbc:mysql://example.com/:3306/?user=admin&password=admin");
8578 8576 ``` ```
8579 8577
8580 that contains three tables (`DISK`, `BOOK` and `VINYL`), each with `Title` and `Price` attributes.
8578 The schema contains three tables (`DISK`, `BOOK` and `VINYL`), each with `Title` and `Price` attributes.
8581 8579 The compiled version is then shared with customers all around the world. The compiled version is then shared with customers all around the world.
8582 8580
8583 8581 You can find a program in a compilable state at `code/java/InsecureProgram.java` that connects to localhost, if you want to test it. You can find a program in a compilable state at `code/java/InsecureProgram.java` that connects to localhost, if you want to test it.
 
... ... Problem (Insecure Java Programming) +.#insecure_java
8586 8584 Question -.# Question -.#
8587 8585 ~ ~
8588 8586
8589 The authors of this program believe that the top-secret title of the next disk by a secret group will not be accessible to the user of this program, because its price is set to `NULL` in the `DISK` table.
8587 The authors of this program believe that the top-secret title of the next disk by a secret group will not be accessible to the user of this program because its price is set to `NULL` in the `DISK` table.
8590 8588 Prove them wrong. Prove them wrong.
8591 8589
8592 8590 Question -.# Question -.#
 
... ... Solution to [%D %n (%T)](#problem:insecure_java)
8603 8601 @problem:insecure_java -- Solution to Q. -.# @problem:insecure_java -- Solution to Q. -.#
8604 8602 ~ ~
8605 8603
8606 This program is vulnerable to `SQL` injection. A user entering "DISK" followed by `0 OR PRICE IS NULL OR PRICE IS NOT NULL` would have access to all the entries, no matter their price tag or lack of absence thereof.
8604 This program is vulnerable to `SQL` injection. A user entering "DISK" followed by `0 OR PRICE IS NULL OR PRICE IS NOT NULL` would have access to all the entries, no matter their price tag or lack of one.
8607 8605
8608 8606 @problem:insecure_java -- Solution to Q. -.# @problem:insecure_java -- Solution to Q. -.#
8609 8607 ~ ~
 
... ... Solution to [%D %n (%T)](#problem:insecure_java)
8613 8611 - Disclosing the name of the tables to the user (DISK, BOOK and VINYL). It would be preferable to use some other name in the program. - Disclosing the name of the tables to the user (DISK, BOOK and VINYL). It would be preferable to use some other name in the program.
8614 8612 - Not asking explicitly for a secure connection is probably not a good idea. Using the default port can sometimes be problematic as well. - Not asking explicitly for a secure connection is probably not a good idea. Using the default port can sometimes be problematic as well.
8615 8613 - Reading a figure as a string is a bad idea, since the user can try to manipulate the content of that field. The datatype read in the application should match the datatype we are trying to get. - Reading a figure as a string is a bad idea, since the user can try to manipulate the content of that field. The datatype read in the application should match the datatype we are trying to get.
8616 - Having `admin` / `admin` as a login / password is unforgivable. They should be changed. And, at least, the application should not connect to the database with admin rights!
8614 - Having `admin` / `admin` as a login / password is unforgivable. The login and password should be changed. And, at least, the application should not connect to the database with admin rights!
8617 8615 - Giving the credentials in the source code is not a good idea. The application should connect to another application, hosted on the the server-side, that performs the connection to the database. Refer e.g. to <https://security.stackexchange.com/q/229954> for explanations on why users should not be allowed to connect directly to your database. - Giving the credentials in the source code is not a good idea. The application should connect to another application, hosted on the the server-side, that performs the connection to the database. Refer e.g. to <https://security.stackexchange.com/q/229954> for explanations on why users should not be allowed to connect directly to your database.
8618 - Not using prepared statement, is a huge mistake. This can lead to `SQL` injection like the one we saw above.
8616 - Not using a prepared statement is a huge mistake. This can lead to `SQL` injection like the one we saw above.
8619 8617
8620 8618
8621 8619 # Presentation of NoSQL # Presentation of NoSQL
 
... ... What does it imply?
9089 9087
9090 9088 Exercise +.#denormalization Exercise +.#denormalization
9091 9089
9092 : What is denormalization? When could that be useful?
9090 : What is denormalization? When could it be useful?
9093 9091
9094 9092 Exercise +.#mismatch Exercise +.#mismatch
9095 9093
9096 : What is the (object-relational) impedance mismatch? Is it an issue that cannot be overcome?
9094 : What is the object-relational impedance mismatch? Is it an issue that cannot be overcome?
9097 9095
9098 9096
9099 9097 Exercise +.# Exercise +.#
9100 9098 ~ ~
9101 9099
9102 For each of the following notion, indicate if they are usually an attribute of NoSQL or of "traditional" `SQL`:
9100 For each of the following notions, indicate if they are usually an attribute of NoSQL or of "traditional" `SQL`:
9103 9101
9104 9102 ||||| |||||
9105 9103 |-------:|:-------------:|---|---|---| |-------:|:-------------:|---|---|---|
 
... ... Exercise +.#
9111 9109 ## Solution to Exercises {-} ## Solution to Exercises {-}
9112 9110
9113 9111 Solution +.# Solution +.#
9114 ~ It is the task of picking the right DBMS for the task, and to involve multiple DBMS in a single application. Yes, it is useful.
9112 ~ It is the act of picking the right DBMS for the task and involving multiple DBMS's in a single application. Yes, it is useful.
9115 9113 Per [wikipedia](https://en.wikipedia.org/wiki/Polyglot_persistence), "Polyglot persistence is the concept of using different data storage technologies to handle different data storage needs within a given software application." Per [wikipedia](https://en.wikipedia.org/wiki/Polyglot_persistence), "Polyglot persistence is the concept of using different data storage technologies to handle different data storage needs within a given software application."
9116 9114
9117 9115 Solution +.# Solution +.#
9118 ~ That a table can contain documents, or tuples, with different attributes.
9119 It implies more responsabilities.
9116 ~ "Schemaless" means hat a table can contain documents, or tuples, with different attributes.
9117 It implies more responsibilities.
9120 9118
9121 9119 Solution +.# Solution +.#
9122 ~ To duplicate data about other entities in some entities.
9120 ~ Denormalization is to duplicate data about other entities in some entities.
9123 9121 It is useful when joining is expensive. It is useful when joining is expensive.
9124 9122
9125 9123 Solution +.# Solution +.#
9126 ~ Data-base and object-oriented principles are different and it requires work to make them work together.
9127 This correspondance, or matching, can be implemented in the application, or lead to the design of new DBMS.
9124 ~ Database and object-oriented principles are different and it requires work to make them work together.
9125 This correspondance, or matching, can be implemented in the application, or lead to the design of a new DBMS.
9128 9126
9129 9127 Solution +.# Solution +.#
9130 9128 ~ ~
 
... ... Solution +.#
9138 9136 ## Problems {-} ## Problems {-}
9139 9137
9140 9138 Problem (Explaining NoSQL) +.#explainNosql Problem (Explaining NoSQL) +.#explainNosql
9141 ~ "NoSQL" used to mean "Non `SQL`", but was retro-actively given the meaning "Not Only `SQL`." Below, write a short essay that explains 1. What motivated the "Non `SQL`" approach, 2. What is the meaning of "Not Only `SQL`", 3. The benefits and limits of the relational approach.
9139 ~ "NoSQL" used to mean "Non `SQL`", but was retro-actively given the meaning "Not Only `SQL`." Below, write a short essay that explains:
9140 #. What motivated the "Non `SQL`" approach.
9141 #. What is the meaning of "Not Only `SQL`."
9142 #. The benefits and drawbacks of the relational approach.
9142 9143
9143 9144 --- ---
9144 9145
 
... ... Problem (ER Diagram from XML File -- Customer) +.#xmltoercustomer
9158 9159 Problem (ER Diagram from XML File -- Award) +.#xmltoeraward Problem (ER Diagram from XML File -- Award) +.#xmltoeraward
9159 9160 ~ ~
9160 9161
9161 Find below a (mashup) of actual data from the National Science Foundation (courtesy of <https://www.nsf.gov/awardsearch/download.jsp>):
9162 Find below a mashup of actual data from the National Science Foundation (courtesy of <https://www.nsf.gov/awardsearch/download.jsp>):
9162 9163
9163 9164 ```{.xml .numberLines include=code/xml/NSFAward.xml} ```{.xml .numberLines include=code/xml/NSFAward.xml}
9164 9165 ``` ```
9165 9166
9166 It contains information about one particular award, awarded to an institution on behalf of two researchers.
9167 It contains information about one particular award that was awarded to an institution on behalf of two researchers.
9167 9168 Quoting the [National Science Foundation](https://www.nsf.gov/about/research_areas.jsp) (NSF): Quoting the [National Science Foundation](https://www.nsf.gov/about/research_areas.jsp) (NSF):
9168 9169
9169 9170 > NSF is divided into the following seven directorates that support science and engineering research and education:…. Each is headed by an assistant director and each is further subdivided into divisions like … > NSF is divided into the following seven directorates that support science and engineering research and education:…. Each is headed by an assistant director and each is further subdivided into divisions like …
9170 9171
9171 From this `xml` file and the information given above, draw a ER diagram for NSF's awards.
9172 Do not hesitate to comment on the choices you are making, and on what justifies them.
9172 From this `xml` file and the information given above, draw an ER diagram for NSF's awards.
9173 Do not hesitate to comment on the choices you are making and on what justifies them.
9173 9174
9174 9175 ## Solutions to Selected Problems {-} ## Solutions to Selected Problems {-}
9175 9176
9176 9177 Solution to [%D %n (%T)](#problem:xmltoercustomer) Solution to [%D %n (%T)](#problem:xmltoercustomer)
9177 9178 ~ ~
9178 9179
9179 It should be clear that 3 entities are present in this file: Customer, Order, and Product.
9180 A product can be part of an order in a certain quantity, and a customer can pass 0 or more orders.
9181 Some attributes are naturally good primary keys (they are named "ID"), and some attributes seems to be optional ("Caution", or "Material"), but should still be given an attribute.
9180 It should be clear that three entities are present in this file: Customer, Order, and Product.
9181 An order can contain a certain quantity of a product, and a customer can pass 0 or more orders.
9182 Some attributes are natural primary keys (they are named "ID" in the diagram below), and some attributes seems to be optional ("Caution", or "Material"), but should still be made an attribute.
9182 9183
9183 9184 Put together, this gives the following diagram: Put together, this gives the following diagram:
9184 9185
9185 9186 ![](fig/er/customers) ![](fig/er/customers)
9186 9187 \ \
9187 9188
9188 We made further assumptions: an order cannot be empty (transcribed by the total constraint on CONTAINS), an order does not exist if it was not passed by a customer (transcribed by the fact that ORDER is a weak entity), which also implies that an order cannot be passed by more than one customer.
9189 Note that the same product cannot be present "twice" (with the equal or different quantities) in an order: an order can contains only once a particular product in any quantity, implying that if an order had 2 of a product A, and 3 of the same product A, those two information should be merged in the fact that an order contains 5 of product A.
9190 This is enforced by the cardinality ratio of 1 in the CONTAINS relationship.
9189 We made further assumptions: an order cannot be empty (transcribed by the total constraint on CONTAINS), and an order does not exist if it was not passed by a customer (transcribed by the fact that ORDER is a weak entity), which also implies that an order cannot be passed by more than one customer.
9190 Note that the same product cannot be present "twice" (with the equal or different quantities) in an order: an order can contain a particular product only once in any quantity, implying that if an order had two of the product A, and three of the same product A, then those two quantities of A should be merged so that an order contains five of this product A.
9191 This is enforced by the cardinality ratio of `1` in the CONTAINS relationship.
9191 9192
9192 Of course, other choices were possible.
9193 Of course, other choices are possible.
9193 9194
9194 9195 Solution to [%D %n (%T)](#problem:xmltoeraward) Solution to [%D %n (%T)](#problem:xmltoeraward)
9195 9196 ~ ~
9196 9197
9197 Two entities are easy to distinguish: RESEARCHER (for "Investigator"), INSTITUTION.
9198 The status of the the content between the `<Organization>` tags is less clear: apparently, an organization has a code, and is made of two parts, a Directorate and a Division.
9198 Two entities are easy to distinguish: RESEARCHER (for "Investigator") and INSTITUTION.
9199 The status of the the content between the `<Organization>` tags is less clear; apparently, an organization has a code, and is made of two parts: a Directorate and a Division.
9199 9200 Using the quote, we know that a Division should be a part of exactly one Directorate, and that a Directorate has an assistant director. Using the quote, we know that a Division should be a part of exactly one Directorate, and that a Directorate has an assistant director.
9200 But what is the status of that "Organization": is it subsumed by the Directorate, is it orthogonal?
9201 But what is the status of that "Organization"? Is it subsumed by the Directorate or is it orthogonal?
9201 9202 We decide to create an entity for it, but its precise role should be clarified. We decide to create an entity for it, but its precise role should be clarified.
9202 9203 The relationship between Division and Directorate is clear, but, once again, the relationship between Division and Organization could have any constraint, we can not really infer that information from the document. The relationship between Division and Directorate is clear, but, once again, the relationship between Division and Organization could have any constraint, we can not really infer that information from the document.
9203 9204
9204 The next difficulty is the status of the award in itself: should it be a relationship with many attributes, between the RESEARCHER and INSTITUTION entities?
9205 The issue with this approach is that an award can have multiple investigators, as shown in the example, and that this number can vary: hence, fixing the arity and constraints on the relationship will be difficult.
9206 We could have a relation of arity 2, and "duplicate it" if multiple researchers are involved in the same grant, but that seems like a poor choice (since all the information about the grant will need to be duplicated).
9207 Hence, it seems more reasonnable to make the award an entity.
9205 The next difficulty is the status of the award itself: should it be a relationship with many attributes, between the RESEARCHER and INSTITUTION entities?
9206 The issue with this approach is that an award can have multiple investigators, as shown in the example, and that this number can vary. Hence, fixing the arity and constraints on this relationship will be difficult.
9207 We could have a relation of arity `2`, and "duplicate it" if multiple researchers are involved in the same grant, but that seems like a poor choice (since all the information about the grant will need to be duplicated).
9208 Therefore, it seems more reasonnable to make the award an entity.
9208 9209
9209 9210 How should we connect the AWARD entity with the RESEARCHER and INSTITUTION entities? How should we connect the AWARD entity with the RESEARCHER and INSTITUTION entities?
9210 A ternary relation has some drawbacks, since it would require some duplication when multiple investigators are working on the same award.
9211 Hence, having one binary relationship between the award and the institution, and one binary relationship between the award and the researcher (that furthermore specifiy the role of the researcher for that particular award), seems like a safer choice.
9212 An award must be awarded to at least one researcher and one institution, but we do not know if there is a maximum number of institution that can obtain the same award, so it is better not to restrict it.
9213 Whenever there should be a relationship between the researcher and the institution is up in the air: we do not know if a researcher has to work for an institution to get a grant, nor if getting a grant for an institution means that you work for it, so it's probably better to refrain from adding such a relationship.
9211 A trinary relation has some drawbacks, since it would require some duplication when multiple investigators are working on the same award.
9212 Instead, having one binary relationship between the award and the institution, and one binary relationship between the award and the researcher (that specifies further the role of the researcher for that particular award), seems like a safer choice.
9213 An award must be awarded to at least one researcher and one institution, but we do not know if there is a maximum number of institutions that can obtain the same award, so it is better not to restrict this arity.
9214 Whether there should be a relationship between the researcher and the institution is up in the air; we do not know if a researcher has to work for an institution to get a grant, nor if getting a grant for an institution means that you work for it, so it is probably better to refrain from adding such a relationship.
9214 9215
9215 Most of the attributes are straightforward, once we noticed that "Role" was an attribute of a relationship, and not of an entity.
9216 Most of the attributes are straightforward once we see that "Role" is an attribute of a relationship, not of an entity.
9216 9217
9217 9218 All together, this gives the following diagram: All together, this gives the following diagram:
9218 9219
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/caubert/CSCI_3410

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/caubert/CSCI_3410

Clone this repository using git:
git clone git://git.rocketgit.com/user/caubert/CSCI_3410

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main