CSC 369: Distributed Computing
Spring 2020

Instructor: Alex Dekhtyar, dekhtyar@calpoly.edu, 14-210

Office Hours:

When		Who	Where
Monday	1:10pm - 2:00pm	Alex	Our Lecture Zoom
Tuesday	1:10pm - 3:00pm	Alex	Office Hours Zoom
Friday	1:10pm - 2:00pm	Alex	Our Lecture Zoom

Note: Zoom links are not shown on this page to prevent web crawlers from finding them. They were emailed to you, and will be available upon an email or Slack request

Additional appoinments: send email.

News and Notes

• On lecture notes and slides: Lecture notes is how I usually convey a written record of what is discussed in class. They have been developed over time. Some notes are terse, some may be overly verbose. This quarter, to provide visuals during our Zoom lectures I use slides. These are meant as a replacement of whatever whiteboard work I do during the class. Also, they provide me with a plan of discussion - my usual, more free-form style works poorly over zoom.
  [April 12, 2020]

• The CAP Theorem lecture notes have been added. Also, the Week 1 Canvas page has Lab 1 output, although there have been some issues with the published files.
  [April 12, 2020]

• Canvas site is now up. Because Canvas is restricted access, some items - video recordings of classes, Zoom and Slack links, for example will only be found there. On the other hand, course handouts and other open access materials are easier to use when they are outside the login protection wall.
  [April 8, 2020]

• This course is taught in synchronous distance mode. This means that all content will be delivered during the scheduled lecture time (MWF 12:10-1:00pm), while lab periods are reserved for individual and pair programming work on the assignments - the same way in-person lab periods are utilized for this course.

  All lectures will be recorded and the recordings, together with raw transcripts will be made available for perusal via our Canvas site. [April 5, 2020]

• Welcome to CSC 369. The page is now up.

Course Materials

Labs

Lab 1	Due: April 13	JSON Manipulation	Postscript	PDF	Lab Data (daily.json)	Source	[April 5, 2020]
Lab 2	Due: April 17	MongoDB: first steps	Postscript	PDF	Lab Data (daily.json)	Source	[April 13, 2020]
Lab 3	Due: April 24	MongoDB aggregation pipelines	Postscript	PDF			[April 21, 2020]
Lab 4	Due: May 10	MongoDB Application		PDF		Lab Info	[April 28, 2020]
Lab 5	Due: May 6	First Hadoop Program	Postscript	PDF		Lab Info	[May 4, 2020]
Lab 6	Due: May 18	Hadoop Programs	Postscript	PDF			[May 8, 2020, 2019]
Lab 7	Due: June 5 June 9	Hadoop Research Mini-project	Postscript	PDF			[May 20, 2020]
Lab 8-1	Due: June 9	Spark		PDF			[May 30, 2020]
Lab 8-1	Due: June 9	Spark Data Frames		PDF	lab08.py	README	[June 4, 2020]
Optional Mini-Project	Due: June 14 (noon)	Distribued Computing with Real Data		PDF			[June 6, 2020]

JSON Resources

Sample Code

MongoDB Resources

April 4, 2020

General welcome, and MongoDB login process

Slides

MongoDB Documentation

MongoDB 4.2 Documentation	HTML
mongo shell	HTML
Create, Read, Update, Delete (CRUD)	HTML
db..find()	HTML
Aggregation Pipeline	Overview,	db.collection.aggregate()	Aggregation pipeline stages

Python MongoDB API

PyMongo 3.10.1 Documentation	HTML
Tutorial	HTML
Authentication example	HTML
Aggregation pipeline example	HTML
Additional examples	HTML

Code and Queries

MongoDB Python API example

example.py

example.out (example.py output)

Hadoop Resources