caubert / CSCI_3410 (public) (License: CC BY 4.0) (since 2018-05-16) (hash sha1)
Material for Database Class.
Clone URLs: https://rocketgit.com/user/caubert/CSCI_3410 ssh://rocketgit@ssh.rocketgit.com/user/caubert/CSCI_3410 git://git.rocketgit.com/user/caubert/CSCI_3410
master
List of commits:
Subject Hash Author Date (UTC)
Edits in intro, converted an image to a figure for example of class diagram. 1ff4ef2f7f44ece81972a6c77e9f6654c144fcdc aubert@math.cnrs.fr 2020-05-14 22:12:38
Adding example file, to test installation. 5ed35e64a4e4dc60c888358bde54594999aab34d aubert@math.cnrs.fr 2020-05-13 19:03:41
Minor adjustments to the syllabus. dca6f12d9d3d561755991029f2d53bf9823a8cea aubert@math.cnrs.fr 2020-05-13 19:02:41
Worked on reverse-engineering section. 34aeba178fe553c08b4fb3738523be293704e09d aubert@math.cnrs.fr 2020-05-13 18:52:09
Cleaning images. ac02dcf8ffbbbd24139491e5d1e349ace782401e aubert@math.cnrs.fr 2020-05-13 17:40:43
Started to spell-check… 799dc31237aa41176687f37306ca4092a7760f89 aubert@math.cnrs.fr 2020-05-13 17:17:06
Added foldable toc to readme and co. 07854a4e108c91c55a40fbd10ed84336b885b93f aubert@math.cnrs.fr 2020-05-13 16:51:17
Added final exam. fb37cf99312795016ddbc2e859b055051bcedd13 aubert@math.cnrs.fr 2020-05-08 21:11:57
Refined ccs style + added foldable toc. 95b1663a7b8294dd401d796fb545a67bddba0a01 aubert@math.cnrs.fr 2020-05-07 19:02:27
Refined ccs style + added foldable toc. c208eee4f769e99415efa57a7f03081db87ec3db aubert@math.cnrs.fr 2020-05-07 18:02:53
Fixing indent in xml and notes. ace8f38f8e0374005d143ebaa32d47a209fb5a48 aubert@math.cnrs.fr 2020-04-27 18:42:28
NoSQL application detailled. f2030de09eb58462e04935ec83c71d3c1fc72469 aubert@math.cnrs.fr 2020-04-27 18:28:02
Added automatic anchors. 2ddb93394121bcf95f91bfed8b447966925afa6f aubert@math.cnrs.fr 2020-04-27 05:35:14
added missing file 480b503949518eb51fd324a6f6bbb8f6d5643f0d aubert@math.cnrs.fr 2020-04-24 02:11:09
Worked on MongoDB section. 70f041d53a14bbb5ace0a4ef40f3e67eca285309 aubert@math.cnrs.fr 2020-04-22 19:18:55
Edited KNOWN_BUGS. b90ca71b498016b17963373ead24873781181bfd aubert@math.cnrs.fr 2020-04-22 05:12:14
Minor edits on narrative. d927955c1269ff1b110057577e50705fd41f4e2d aubert@math.cnrs.fr 2020-04-22 05:07:48
Worked on migrating code to snippets. 1ce6f6ba150a1cace57445330801e8b17f6c3044 aubert@math.cnrs.fr 2020-04-22 04:15:37
Edited Sol of 4.28, Pb and Sol of 4.29, checked Pb 4.30, edited Pb 4.31, edited the DB Applications Resources section, checked Pb 5.1, edited Pb 5.2 and Sol 5.2, Pb and Sol of 6.1, Pb 7.1, checked Pb 7.2, edited Sol 7.2, Pb and Sol of Pb 7.3, checked Ex 7.1 and edited its solution, checked Ex 7.2 and edited its solution, edited Ex and Sol of 7.3, Ex and Sol of 7.4 edited Ex 7.5 and checked its solution, edited Ex and Sol of 6.1, Ex and Sol of 6.2, checked Ex 6.3 and edited its solution, checked Ex 5.1 and edited its solution, checked Ex 5.2 and edited its solution, edited Ex and Sol of 5.3, checked Ex and Sol of 5.4, checked Ex 5.5 and edited its solution, and edited Ex and Sol of 5.6. 6580a85df655eddd149bb5531a0615ddfe9f810e Crystal 2020-04-22 03:02:20
Disabled Scrolling of sql files in html and enabled resize option 87d4ef4185fe48a7503724eb98a6c486c351ddee pveeral@augusta.edu 2020-04-21 15:49:36
Commit 1ff4ef2f7f44ece81972a6c77e9f6654c144fcdc - Edits in intro, converted an image to a figure for example of class diagram.
Author: aubert@math.cnrs.fr
Author date (UTC): 2020-05-14 22:12
Committer name: aubert@math.cnrs.fr
Committer date (UTC): 2020-05-14 22:12
Parent(s): 5ed35e64a4e4dc60c888358bde54594999aab34d
Signer:
Signing key:
Signing status: N
Tree: b685d19d5ec7781238863996b6c5fc77eeb921d7
File Lines added Lines deleted
notes/fig/uml/rectangle.tex 23 0
notes/img/class_diagram.png 0 0
notes/lectures_notes.md 29 32
File notes/fig/uml/rectangle.tex added (mode: 100644) (index 0000000..823c2f6)
1 \documentclass[border=20pt]{standalone}
2 \input{template.def}
3
4 \begin{tikzpicture}
5 \begin{class}[text width=7cm]{Rectangle}{0,0}
6 \attribute{- length : int}
7 \attribute{- width : int}
8 \operation{+ SetLength(int length) : void}
9 \operation{+ GetLength() : int}
10 \operation{+ ComputeArea() : int}
11 \end{class}
12 \begin{class}{Cube}{11, -0.63}
13 \inherit{Rectangle}
14 \attribute{- height : int}
15 \operation{+ ComputeVolume(): int}
16 \end{class}
17 \node at (2.6, -.3) {\textcolor{green}{\emph{Class Name}}};
18 \node at (2.6, -1.3) {\textcolor{green}{\emph{Attributes}}};
19 \node at (2, -2.7) {\textcolor{green}{\emph{Methods, Operations}}};
20 \node at (6, -1.3) {\textcolor{green}{\emph{Extends}}};
21 \end{tikzpicture}
22
23 \end{document}
File notes/img/class_diagram.png deleted (index e0324eb..0000000)
File notes/lectures_notes.md changed (mode: 100644) (index 6178d98..2974fcf)
... ... _I would like here to include some of the softwares I believe could make your se
274 274
275 275 There is a good chance that any programming language you can think of is [Turing complete](https://en.wikipedia.org/wiki/Turing-completeness). There is a good chance that any programming language you can think of is [Turing complete](https://en.wikipedia.org/wiki/Turing-completeness).
276 276 Actually, even some of the extremely basic tools you may be using [may be Turing complete](https://www.gwern.net/Turing-complete). Actually, even some of the extremely basic tools you may be using [may be Turing complete](https://www.gwern.net/Turing-complete).
277 However, being complete does not mean being good at any task: it just mean that any computable problem can be solved, but does not imply anything in terms of efficiency, comfort, or usability.
277 However, being complete does not mean being good at any task: it just means that any computable problem can be solved, but does not imply anything in terms of efficiency, comfort, or usability.
278 278
279 279 In theory, pretty much any programming language can be used to In theory, pretty much any programming language can be used to
280 280
 
... ... In theory, pretty much any programming language can be used to
282 282 - Have accessible catalog describing the metadata, - Have accessible catalog describing the metadata,
283 283 - Support transactions and concurrency, - Support transactions and concurrency,
284 284 - Support authorization of access and update of data, - Support authorization of access and update of data,
285 - Enforce constraints,
285 - Enforce constraints.
286 286
287 287 But to obtain a system that is fast in reading and writing on the disk, convenient to search in the data, and that provides as many "built-in" tools as possible, one should use a specialized tool. But to obtain a system that is fast in reading and writing on the disk, convenient to search in the data, and that provides as many "built-in" tools as possible, one should use a specialized tool.
288 288
289 In this lecture notes, we will introduce one of this tool--the `SQL` programming language-- and the theory underneath it--the relational model--.
289 In those lecture notes, we will introduce one of this tool--the `SQL` programming language-- and the theory underneath it--the relational model--.
290 290 We will also observe that a careful design is a mandatory step before implementing a catalog, and that how good a catalog is can be assessed, and introduce the tools to do so. We will also observe that a careful design is a mandatory step before implementing a catalog, and that how good a catalog is can be assessed, and introduce the tools to do so.
291 Finally, we will discuss how an application interacting with a database can be implemented and secured, and the alternatives to `SQL` offered by the NoSQL approach, as well as the limitations of both models.
291 Finally, we will discuss how an application interacting with a database can be implemented and secured, and the alternatives to `SQL` offered by the NoSQL approach, as well as the limitations and highlights of both models.
292 292
293 293 ## Database ## Database
294 294
295 295 A database (DB) is **a collection of related data**. A database (DB) is **a collection of related data**.
296 296 Data (= information, can be anything, really) + management (= logical organization of the data), through **D**ata**b**ase **M**anagement **S**ystem. Data (= information, can be anything, really) + management (= logical organization of the data), through **D**ata**b**ase **M**anagement **S**ystem.
297 297
298 #. Represent a mini-world, a Universe of Disclosure (UoD).
298 #. Represent a mini-world, a "Universe of Disclosure" (UoD).
299 299 #. Logically coherent, with a meaning. #. Logically coherent, with a meaning.
300 300 #. Populated for a purpose. #. Populated for a purpose.
301 301
 
... ... A DBMS has multiple components, as follows:
307 307
308 308 Note that Note that
309 309
310 - The program can be written in any language, be a web interface, etc. It is sometimes part of the software shipped with the DBMS, but not necessarily.
311 - Most DBMS software include a Command-Line Interface (C.L.I.).
312 - The catalog (or schema, meta-data^[The term "meta-data" has numerous definition ("data about the data"): we use it here to refer to the description of the organization of the data, and not e.g. to statistical data about our data.]) contains the description of how the data is stored, i.e., the datatypes, nature of the attributes, etc.
310 - The program can be written in any language, be a web interface, etc. It is sometimes part of the software shipped with the DBMS, but not necessarily (you can, and we will, develop your own program to interact with the DBMS).
311 - Most DBMS software includes a Command-Line Interface (CLI).
312 - The catalog (or schema, meta-data^[The term "meta-data" has numerous definition ("data about the data"): we use it here to refer to the description of the organization of the data, and not e.g. to statistical data about the data.]) contains the description of how the data is stored, i.e., the datatypes, nature of the attributes, etc.
313 313 - Sometimes, catalog and data are closer than pictured (you can have "self-describing meta-data", that is, they cannot be distinguished). - Sometimes, catalog and data are closer than pictured (you can have "self-describing meta-data", that is, they cannot be distinguished).
314 314
315 315 ## Database Management System (DBMS) ## Database Management System (DBMS)
 
... ... A DBMS contains a *general purpose* software that is used to
323 323
324 324 You can think of a tool to You can think of a tool to
325 325
326 #. specify a storage unit,
327 #. fill it,
328 #. allow to change its content, as well as its organization,
329 #. allow multiple persons to access all or parts of it at the same time.
326 #. Specify a storage unit,
327 #. Fill it,
328 #. Allow to change its content, as well as its organization,
329 #. Allow multiple persons to access all or parts of it at the same time.
330 330
331 331 ## Subtasks ## Subtasks
332 332
333 333 Exactly like a program can have Exactly like a program can have
334 334
335 - a client, that specify the requirements,
335 - clients, that specify the requirements,
336 336 - designers, that define the overall architecture of a program, - designers, that define the overall architecture of a program,
337 337 - programmers, that implement the details of the program, - programmers, that implement the details of the program,
338 338 - testers, that make sure the program is free of bugs, and - testers, that make sure the program is free of bugs, and
 
... ... Exactly like a program can have
340 340
341 341 a DBMS offers multiple (sub)tasks and can be interacted with different persons with different roles. a DBMS offers multiple (sub)tasks and can be interacted with different persons with different roles.
342 342
343 <!--
344 can be used by multiple users for different reasons, and different tasks can be assigned to different members of a project.
345 -->
346 343
347 344 | Role | Task | | Role | Task |
348 345 | --- | ---------- | | --- | ---------- |
 
... ... can be used by multiple users for different reasons, and different tasks can be
352 349 | Programmer | Implement the database, work on the programs that will interface with it | | Programmer | Implement the database, work on the programs that will interface with it |
353 350 | User | Provide, search, and edit the data (usually) | | User | Provide, search, and edit the data (usually) |
354 351
355 <!--
356 #. Maintenance (DB administrator)
357 #. Organization (DB designer)
358 #. Modification, retrieval (end-user)
359 #. Software engineer, web developer, programers, …
360 -->
361
362 In those lecture notes, the main focus will be on design, but we will have to do a little bit of everything, without forgetting which role we are currently playing.
352 In those lecture notes, the main focus will be on design and implementation, but we will have to do a little bit of everything, without forgetting which role we are currently playing.
363 353
364 354 ## Life of a Project ## Life of a Project
365 355
 
... ... You can describe the structure as a collection of relations, and a collection of
448 438
449 439 ### Interactions ### Interactions
450 440
451 - This organization will allow some interactions. For instance, we can obtain the answer to questions like "What is the name of the course whose number is 1301?", "What courses is Kate teaching this semester?", "Does Bob meets the pre-requisite for 2910?", etc. Note that this last query is a bit different, as it forces us to look up information in multiple relations.
441 - This organization will allow some interactions. For instance, we can obtain the answer to questions like
442
443 > "What is the name of the course whose number is 1301?",
444 > "What courses is Kate teaching this semester?",
445 >"Does Bob meets the pre-requisite for 2910?"
446
447 Note that this last query is a bit different, as it forces us to look up information in multiple relations.
452 448 - We should also be able to perform updates, removal, addition of records in an efficient way (using auxiliary files (indexes), optimization). - We should also be able to perform updates, removal, addition of records in an efficient way (using auxiliary files (indexes), optimization).
453 449 - Finally, selection (for any operation) requires care: do we want all the records, some of them, exactly one? - Finally, selection (for any operation) requires care: do we want all the records, some of them, exactly one?
454 450
 
... ... You can describe the structure as a collection of relations, and a collection of
456 452
457 453 Why are the files separated like that? Why are the files separated like that?
458 454 Why do not we store the section with the course with the students? Why do not we store the section with the course with the students?
455 For multiple reasons:
459 456
460 - **Avoiding redundancy** ("data normalization"), or having it controlled,
461 - **Levels of access** (multiple user interface),
462 - And we still have the same usability!
457 - To **avoid redundancy** ("data normalization"), or having it controlled,
458 - To controle multiple **levels of access** (multiple user interface),
459 - Without sacrificing the **usability**!
463 460
464 But need to be careful about **consistency** / **referential integrity**.
461 In separating the datae, we also need to remember to be careful about **consistency** and **referential integrity**, which is a topic we will discuss in detail.
465 462
466 463 ### How Is a Database Conceived? ### How Is a Database Conceived?
467 464
 
... ... But need to be careful about **consistency** / **referential integrity**.
470 467 #. Logical design #. Logical design
471 468 #. Physical design #. Physical design
472 469
473 Gradation, from really abstract specification that is easy to modify, to more solidified description of what needs to be coded.
470 There is a gradation, from really abstract specification that is easy to modify, to more solidified description of what needs to be coded.
474 471 When we will be discussing high-level models, we will [come back to those notions](#interest-for-high-level-design). When we will be discussing high-level models, we will [come back to those notions](#interest-for-high-level-design).
475 472 The global idea is that it is easier to move things around early in the conception, and harder once everything is implemented. The global idea is that it is easier to move things around early in the conception, and harder once everything is implemented.
476 473
 
... ... Wide, powerful, but also intimidating.
5810 5807
5811 5808 You know UML from object-oriented programming language: You know UML from object-oriented programming language:
5812 5809
5813 ![Class Diagram](img/class_diagram.png){width=90%}
5810 ![](fig/uml/rectangle)
5814 5811
5815 That's a class diagram, there are other types of diagrams, they are not unrelated!
5812 That is an example of a class diagram (with class name, attributes and operators, as well as a particular way to represent that a class extends another), there are other types of diagrams, they are not unrelated!
5816 5813 For instance, using communication diagrams, deployment diagrams, and state chart diagrams, you can collect the requirements needed to draw a class diagram! For instance, using communication diagrams, deployment diagrams, and state chart diagrams, you can collect the requirements needed to draw a class diagram!
5817 5814 They each offer a viewpoint on a software that will help you in making sure the various pieces will fit together: it is a tool commonly used in software engineering, and useful in database design. They each offer a viewpoint on a software that will help you in making sure the various pieces will fit together: it is a tool commonly used in software engineering, and useful in database design.
5818 5815
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/caubert/CSCI_3410

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/caubert/CSCI_3410

Clone this repository using git:
git clone git://git.rocketgit.com/user/caubert/CSCI_3410

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main