Project
Week 6 - Beta Prototype Demos
26/10/11 16:27 Filed in: 480
During the lab on Thursday this week and Tuesday next week, I’d like to see demonstrations of your team’s beta prototype. If you want to, you can put together a brief presentation and do it in front of the whole class; otherwise I’ll visit the teams and go over their demo with them.
Clarification of Milestone Due Dates and Peer Team Evaluations
24/10/11 14:29 Filed in: 480
I’ve labeled the milestones by the week when they are due, and I expect them to be completed by the end of the day on Thursday of that week. So Milestone Week 6 is due this week, on Thursday, by 23:59:59. If your team is a few days behind, you can request an extension for the milestone, which I usually grant. The extension is usually over the weekend, and needs to be requested via email.
The evaluation process involves at a minimum the review of the documentation produced by the development team. If possible, the evaluation team should download and install prototypes and try out the system, preferably on the basis of the evaluation plan put together by the development team in Milestone Week 4. At the discretion of the evaluation team, they can also meet with the development team to discuss the project, or to see the current status of the system under development. This is especially advisable if the prototype or final system requires special software or hardware not available to the development team
The evaluation is due one week after the respective milestone. However, I won’t penalize the evaluation team if the development team delivers the milestone late. So as of today (Mon, Week 6) you should have the evaluation of Milestone Week 4 completed.
The evaluation process involves at a minimum the review of the documentation produced by the development team. If possible, the evaluation team should download and install prototypes and try out the system, preferably on the basis of the evaluation plan put together by the development team in Milestone Week 4. At the discretion of the evaluation team, they can also meet with the development team to discuss the project, or to see the current status of the system under development. This is especially advisable if the prototype or final system requires special software or hardware not available to the development team
The evaluation is due one week after the respective milestone. However, I won’t penalize the evaluation team if the development team delivers the milestone late. So as of today (Mon, Week 6) you should have the evaluation of Milestone Week 4 completed.
Peer Evaluations for Project Teams
13/10/11 12:51
I’m working on the assignment of evaluation teams for the project (see http://users.csc.calpoly.edu/~fkurfess/Courses/480/F11/Project/Project/Team-Evaluations.html).As a team, please look at the project list below or at http://users.csc.calpoly.edu/~fkurfess/Courses/480/F11/Project/Project/Teams.html and select your top two choices of teams you'd like to evaluate. I will then assign the evaluation teams.
F.K.
F.K.
| Section 1 | ||||
| Team 2: Social Intelligence | ||||
| Team 3: Dreamteam | ||||
| Team 4: Board Game AI | ||||
| Team 5: SemanticMediaWiki | ||||
| Team 6: PacMan | ||||
| Team 7: NLP and Information Extraction |
| Section 3 | ||||
| Team 10: Optical Character Recognition | ||||
| Team 11: Stealth Game | ||||
| Team 12: Texas Hold Em | ||||
| Team 13: Evolutionary Algorithms | ||||
| Team 14: PacMan | ||||
| Team 15: Translation of Asian Languages to English |
Project Milestone Feedback Templates
13/10/11 12:50
I’ve added the templates I'm using for the evaluation of the project milestones toh the Web page describing the project, http://users.csc.calpoly.edu/~fkurfess/Courses/480/F11/Project/Project.html in the "Grading" part at the bottom of the page.
For Milestone Week 4, the emphasis is on the aspects below
User Feedback
initial reactions, criticism, suggestions, etc. from users of the alpha prototype or to design sketches
possible design modifications, changes in implementation
System Design
overview of the functional parts of the system
block diagram, class hierarchy
Prototype and Implementation
restrictions in functionality
implementation language, development tools
instructions for installation and basic use
Evaluation Plan
description of evaluation methods
detailed instructions for performing the evaluation
explicit evaluation criteria (questions, metrics)
For Milestone Week 4, the emphasis is on the aspects below
User Feedback
initial reactions, criticism, suggestions, etc. from users of the alpha prototype or to design sketches
possible design modifications, changes in implementation
System Design
overview of the functional parts of the system
block diagram, class hierarchy
Prototype and Implementation
restrictions in functionality
implementation language, development tools
instructions for installation and basic use
Evaluation Plan
description of evaluation methods
detailed instructions for performing the evaluation
explicit evaluation criteria (questions, metrics)
Project Documentation on TRAC Wikis
04/10/11 14:42
Over the weekend I had a first look at the project documentation on the team wikis. While I received a few requests for an extension of the deadline of last Thursday, I was quite disappointed to see that most teams do not have anything on their TRAC Wikis yet.
I will grade the Milestone Week 2 documents on Friday, so you need to make sure that your TRAC wiki contains at least the following by Thursday, midnight:
- Title of you project, optionally a team name
- list of the team members
- short description of the project
- comparison against existing related systems
- features, requirements, and evaluation criteria
- schedule
For reference purposes, you can look at last year’s team with the highest score for their documentation, Project Deuce.
My general policy is to let teams revise each document once, but the maximum improvement of their score will be half of the difference between the overall maximum score and their score in the first version. So if a team gets a score of 4 out of 10, the best they can hope for after revisions is (10-4)/2 = 3, resulting in an final score of 4+3 - 7.
I will grade the Milestone Week 2 documents on Friday, so you need to make sure that your TRAC wiki contains at least the following by Thursday, midnight:
- Title of you project, optionally a team name
- list of the team members
- short description of the project
- comparison against existing related systems
- features, requirements, and evaluation criteria
- schedule
For reference purposes, you can look at last year’s team with the highest score for their documentation, Project Deuce.
My general policy is to let teams revise each document once, but the maximum improvement of their score will be half of the difference between the overall maximum score and their score in the first version. So if a team gets a score of 4 out of 10, the best they can hope for after revisions is (10-4)/2 = 3, resulting in an final score of 4+3 - 7.
Pacman vs Wumpus World
03/10/11 10:56
Over the last few years, we’ve used the Wumpus World environment (BotEnvironment and WumpusEnvironment) for several of the lab and homework activities. While it works, it has its quirks, which is not surprising considering that it’s essentially an enhanced extra-credit assignment done by Matt Colón, a former 480 student.
I just saw that the Stanford AI course (which will also be offered online for free) uses a Pacman environment written in Python for the Search algorithms assignment. If anybody is interested in doing this instead of the Wumpus World, let me know. What’s especially appealing to me, of course, is the auto-grading option that they apparently use. While I don’t know this, I assume they will use the auto-grade option for the online version as well. To get the assignment graded by their system, you’ll have to enroll in the online course, and select the “advanced” option.
This may be especially interesting for the teams working on a Pacman team project anyway.
I just saw that the Stanford AI course (which will also be offered online for free) uses a Pacman environment written in Python for the Search algorithms assignment. If anybody is interested in doing this instead of the Wumpus World, let me know. What’s especially appealing to me, of course, is the auto-grading option that they apparently use. While I don’t know this, I assume they will use the auto-grade option for the online version as well. To get the assignment graded by their system, you’ll have to enroll in the online course, and select the “advanced” option.
This may be especially interesting for the teams working on a Pacman team project anyway.
Project TRAC Wikis Available
23/09/11 17:48
Our sysadmins created a number of TRAC wikis for us, to be used for the team projects. I want you to use them mostly for the project documentation, but you can also use their SVN functionality as a code repository. On Tuesday, we’ll assign the team numbers, and I’ll add one team member to the wiki. This person then can add the rest of the team. If you can’t wait until Tuesday, send me an email, and I’ll do it earlier. Please include your section (morning or afternoon) and the project title in that email.
My strong preference is for all teams to use the TRAC Wiki, at least for the documentation. This is mainly for two reasons: Persistence (documents on your personal Cal Poly Web page, for example, will disappear when you graduate), and consistency, which makes it easier for me to find a particular piece of documentation. If you have a good argument for using something else, I’m open to it, but please discuss it with me first.
For each team wiki, I've created a group "Members" with SVN_READ, SVN_WRITE and TRAC_ADMIN privileges.
To add your team members to the wiki, do the following:
If you want to exercise tighter controls, you can also create different groups with restricted privileges, and assign them to those groups.
My strong preference is for all teams to use the TRAC Wiki, at least for the documentation. This is mainly for two reasons: Persistence (documents on your personal Cal Poly Web page, for example, will disappear when you graduate), and consistency, which makes it easier for me to find a particular piece of documentation. If you have a good argument for using something else, I’m open to it, but please discuss it with me first.
For each team wiki, I've created a group "Members" with SVN_READ, SVN_WRITE and TRAC_ADMIN privileges.
To add your team members to the wiki, do the following:
- go to the Wiki
- login with your CSC login information (login name, password)
- go the the "Admin" page by clicking on the "Admin" tab , and then "Permissions" in the list on the left.
- enter their login names one by one in the "Add Subject to Group" area to add them to the "Members" group
If you want to exercise tighter controls, you can also create different groups with restricted privileges, and assign them to those groups.
Project Topic Suggestions
20/09/11 10:29
The documentation for the term project is available now, including suggestions for project topics.
Updated Grades on Blackboard
09/11/11 20:38 Filed in: 480
I’ve updated Blackboard with the scores for the following items
* Labs L1 through L7
I just noticed that some of you submitted non-programming labs via Blackboard; I still have to evaluate those.
* Assignments A1 and A2
If you have a score of around 10 for A2, it is only for the first part.
The scores reflect only the programs shown in the lab. I haven’t graded the rest yet.
I changed the maximum score on Blackboard from 33 to 25.
* Project Week 4 and 6 Milestones
I believe this includes all requests for re-evaluation. Let me know if you’ve asked for one, but your score hasn’t changed.
* AI Nugget presentations
I believe I’ve graded all the presentations given so far.
If you’ve given your presentation, but your score is around 60, you probably haven’t submitted your presentation material (slides, Prezi) on the Blackboard Wiki.
* Labs L1 through L7
I just noticed that some of you submitted non-programming labs via Blackboard; I still have to evaluate those.
* Assignments A1 and A2
If you have a score of around 10 for A2, it is only for the first part.
The scores reflect only the programs shown in the lab. I haven’t graded the rest yet.
I changed the maximum score on Blackboard from 33 to 25.
* Project Week 4 and 6 Milestones
I believe this includes all requests for re-evaluation. Let me know if you’ve asked for one, but your score hasn’t changed.
* AI Nugget presentations
I believe I’ve graded all the presentations given so far.
If you’ve given your presentation, but your score is around 60, you probably haven’t submitted your presentation material (slides, Prezi) on the Blackboard Wiki.
Milestone Week 6 Feedback
31/10/11 21:09 Filed in: 480
The feedback for the Week 6 Milestone documents is now on the TRAC wikis for the respective projects. The scope, as with the previous ones, ranges from non-existent to excellent. You can revise your document, but you need to let me know if you’ve made changes relevant to this milestone, and would like a re-evaluation.
Milestone Week 4 Feedback
31/10/11 21:08 Filed in: 480
I’ve posted the feedback for the Week 4 Milestone documents on the TRAC wikis for the respective projects. As with the previous one, quite a few teams have significant gaps in their documentation, and received a low score. You can revise your document, but you need to let me know if you’ve made changes relevant to this milestone, and would like me to re-evaluate the document.
Feedback on Team Work
26/10/11 16:15 Filed in: 480
Around the midpoint of the quarter, I’d like to check if all the teams are doing well in terms of collaboration and work distribution. Please go to the Team Feedback form for this.
The mid-quarter version mostly helps me to determine if there are problems with teams. We’ll do another one at the end of the quarter, and the collective feedback from that one will also affect your team score, albeit to a moderate degree (most students usually get scores around 16-18 out of 20 for this).
The mid-quarter version mostly helps me to determine if there are problems with teams. We’ll do another one at the end of the quarter, and the collective feedback from that one will also affect your team score, albeit to a moderate degree (most students usually get scores around 16-18 out of 20 for this).
Milestone Week 2 Feedback Posted
07/10/11 18:14 Filed in: 480
I’ve posted the feedback for the Week 2 Milestone documents on the TRAC wikis for the respective projects. I’ve added a PDF file as an attachment to the Milestone Week 2 document (or whichever document contains the respective information). Most teams should revise their documents.
Let me know if you want me to re-evaluate your document, and point out the main changes you’ve made. Of course we can also discuss this during the lab time.
Let me know if you want me to re-evaluate your document, and point out the main changes you’ve made. Of course we can also discuss this during the lab time.
Evaluation Teams Assigned
19/10/11 20:41 Filed in: 480
I’ve completed the assignment of an evaluation team for each project. Go to the page with the project teams, look for your team number in the column on the right, and then do the evaluation of the team that is listed on the left. If you want to, you can use the template that I’m using for the milestone evaluations, but you can also select your own scheme.
For me, there are two important aspects to this peer evaluation: First, I’d like to have an extra set of eyes take a look at the project under evaluation. Second, I want the team under evaluation to get additional feedback from someone doing similar work, even if the nature and domain of their project is different.
Put the evaluation on the wiki for your own project by creating a new Wiki page called “Evaluations”. An easy way to do this is to put a link to the new page on one of the existing pages via
For me, there are two important aspects to this peer evaluation: First, I’d like to have an extra set of eyes take a look at the project under evaluation. Second, I want the team under evaluation to get additional feedback from someone doing similar work, even if the nature and domain of their project is different.
Put the evaluation on the wiki for your own project by creating a new Wiki page called “Evaluations”. An easy way to do this is to put a link to the new page on one of the existing pages via
[wiki:Evaluations Evaluations]. From there, you can link to the individual evaluations of the milestones. The deadline for the evaluations is one week after the milestones are due. However, evaluation teams won’t be penalized if documents by the development team are not made available on time. Project Template Available on your Team Wiki
02/10/11 19:06 Filed in: 480
When you create new pages in your wiki, you can select templates from a pull-down menu. Those templates are intended for my courses with team projects, and they contain information on what to put in the respective pages.
Updated Grades on Blackboard
30/11/11 19:13 Filed in: 480
Earlier today, I updated the Blackboard Grade Sheet again. Here’s the current status
* Labs L1 through L9
Up to date as of this afternoon; I still have to evaluate programming labs submitted on Blackboard and not demoed in lab, and some non-programming labs submitted via Blackboard;
* Assignments A1 and A2
Up to date as of this afternoon; I still have to evaluate assignments submitted on Blackboard and not demoed in lab
* Project Week 4 and 6 Milestones
Updated to reflect requests for re-evaluation; I haven’t done the Week 8 Milestones yet
* AI Nugget presentations
Should reflect all the presentations given so far, and materials submitted as of this afternoon.
* Quiz Make-up Questions and Requests for Re-evaluation
Still need to do these; I may also curve the last two (Q8 and Q9) since the averages (66 and 71) were a bit lower than for the earlier ones (around 80), for those students who actually took the quizzes.
* Labs L1 through L9
Up to date as of this afternoon; I still have to evaluate programming labs submitted on Blackboard and not demoed in lab, and some non-programming labs submitted via Blackboard;
* Assignments A1 and A2
Up to date as of this afternoon; I still have to evaluate assignments submitted on Blackboard and not demoed in lab
* Project Week 4 and 6 Milestones
Updated to reflect requests for re-evaluation; I haven’t done the Week 8 Milestones yet
* AI Nugget presentations
Should reflect all the presentations given so far, and materials submitted as of this afternoon.
* Quiz Make-up Questions and Requests for Re-evaluation
Still need to do these; I may also curve the last two (Q8 and Q9) since the averages (66 and 71) were a bit lower than for the earlier ones (around 80), for those students who actually took the quizzes.
Another Round of Grade Updates on Blackboard
05/12/11 21:10 Filed in: 480
I just updated the Blackboard Grade Sheet again. Here’s the current status
* Labs L1 through L10
Up to date as of this evening; I still have to evaluate programming labs submitted on Blackboard and not demoed in lab, and some non-programming labs submitted via Blackboard;
* Assignments A1 and A2
Up to date as of this afternoon; I still have to evaluate assignments submitted on Blackboard and not demoed in lab.
I’ll have to adjust the Blackboard calculation for the reduction of assignments from 3 to 2. So if you didn’t do A3, your overall assignment grade should be better.
* Project Week 4, 6 and 8 Milestones, Final Presentations, Team Evaluation, Team Member Evaluations
Updated, and should be complete except for those students who have not submitted their mutual team member evaluations yet.
My spreadsheet automatically gives you 0 if you haven’t done a self-evaluation; I’ll change this later if you’ve done them for your team members. If you haven’t done any yet, do them as soon as possible.
* AI Nugget presentations
Should reflect all the presentations given so far, and materials submitted as of this afternoon.
* Quiz Make-up Questions and Requests for Re-evaluation
Still need to do these; I may also curve the last two (Q8 and Q9) since the averages (66 and 71) were a bit lower than for the earlier ones (around 80), for those students who actually took the quizzes.
* Labs L1 through L10
Up to date as of this evening; I still have to evaluate programming labs submitted on Blackboard and not demoed in lab, and some non-programming labs submitted via Blackboard;
* Assignments A1 and A2
Up to date as of this afternoon; I still have to evaluate assignments submitted on Blackboard and not demoed in lab.
I’ll have to adjust the Blackboard calculation for the reduction of assignments from 3 to 2. So if you didn’t do A3, your overall assignment grade should be better.
* Project Week 4, 6 and 8 Milestones, Final Presentations, Team Evaluation, Team Member Evaluations
Updated, and should be complete except for those students who have not submitted their mutual team member evaluations yet.
My spreadsheet automatically gives you 0 if you haven’t done a self-evaluation; I’ll change this later if you’ve done them for your team members. If you haven’t done any yet, do them as soon as possible.
* AI Nugget presentations
Should reflect all the presentations given so far, and materials submitted as of this afternoon.
* Quiz Make-up Questions and Requests for Re-evaluation
Still need to do these; I may also curve the last two (Q8 and Q9) since the averages (66 and 71) were a bit lower than for the earlier ones (around 80), for those students who actually took the quizzes.