The final project teams create a custom visualization for a specific data set and set of visualization goals. This includes both an ideal design and an as-built implementation. It also includes formal and informal evaluations of the implementation.
There are three major parts to the project: visualization goals specification, visualization system development, and visualization system evaluation. Each will be described as part of the final written report on the project, which is due April 29th (a draft is due March 27th). The format of the write-up should be in the style of a VisWeek Application Paper (although more detailed in parts than those found in proceedings); projects that reveal particularly fruitful insights have been written up after the class was finished for the IEEE VisWeek conference and other venues.
This is the problem statement. It should include all information relevant to the description of the problem that is being investigated. This portion of the document should be pulled mainly from the problem descriptions in the Spring of 2014, and it is therefore a very small fraction of the total project score. Do at least one more interview with the client to see about additional data sets and questions not listed in the homework, and to get clarification on the relative importance of the questions.
For this project, you do not need to limit yourself to a single design to answer all of the goals. Include as many designs as needed to cover the goals well (note how ImageSurfer had two different visualizations for the question of layout on the membrane as opposed to within the membrane). You can design still images, animations, or interactive tools, use stereoscopic display if it helps improve comprehension, and include haptics if it is helpful.
Describe why the particular display channels are being chosen for this data set and set of tasks. Describe how each question posed in the visualization goals specification is being addressed by the visualization design. What display channels will convey information towards each of the goals (e.g.: height field carries height information and color carries conductance; direct volume rendering conveys density information and haptic display conveys density gradient; geometric model conveys context and slices textured with field-tracing lines convey magnetic field). What sort of interpolation is being done on the data? What is being done about missing values, if any? What legends will be present to help the user quantitatively estimate values? What color map is being used and why?
Also present a second-best design and describe why it would be inferior in terms of how its characteristics support or detract from the visual perception needed to answer each question.
Describe the ideal approaches to be taken to visualize the data. It is important to not let concerns over what you think you will be able to implement constrain your design. The above description should present the ideal visualization design for the data and questions you have, regardless of whether it can be fully implemented during the semester.
Include screen shots of any prototyping you have done on the visualization, or scanned images of visualizations using techniques that are similar to those you would use.
The implementation of the above design into a visualization tool or constructed program.. If it is the case that you will not be able to implement the ideal visualization design this semester, then describe the actual system implemented.
Describe how the design was turned into a working system. What toolkits are being used (ParaView, VTK, Matlab, etc.), and what underlying libraries and languages are being used (openGL, OpenHaptics, C++, etc.). What display hardware and input devices are being used? What computation and image-generation hardware is being used? Provide a high-level description of the program modules and how they work together. List or describe the algorithms that were used or developed for the visualization (marching cubes, Runge-Kutta integration, color space transformations). The level of description should be adequate to enable a computer science student who is not a visualization expert to reproduce the work.
Include screen shots to show features of the the visualization system. To the extent that they would help a future team continuing work on this project, Include lessons learned along the way (blind alleys that you tried before coming to the final implementation, things that worked better than you expected, interesting things you found out that you didn't expect). Include a discussion of how you might implement the system differently the next time if given the chance.
There will be two in-class presentations of the design and evaluation. The second of the two will be included in the grade.
There will be an in-class presentation of the initial design and evaluation plan on April 1st and 3rd; there will be an accompanying written document to be turned in on March 27th. This will include a reminder of the questions being focused on. This will be evaluated, but will not count as part of the grade for the project. It is an opportunity to get feedback from the instructor and the rest of the class to help improve your final design and evaluation.
On April 22nd and 24th, each team will orally present the results of the design, evalaution, and implementation. This will include a reminder of the questions being focused on, together with the visualization design (ideal design) and as-implemented display for each question or set of questions. The team will then report on the results of the formal evaluation and any changes made to the design based on lessons learned during the evaluation.
You can present still images, animations, and/or interactive systems to show your work.
How well does the implementation work in practice?
Formal evaluation on test data: This is not a formal user study (which would involve a number of users performing a set of tasks and include statistical analysis of their performance using the system). It is rather a test of the system by a single person who is not one of the system's authors to see if it meets its goals. It can be performed on either the actual data from the client or on a special data set designed to test its effectiveness (you get more points the more difficult it is to reach the goal using a much simpler visualization or even computational techniques). If it is performed on the actual data set, then there must be an independent way of verifying the accuracy of the answers given; this will be challenging because usually all we will have are visualizations of this data and the questions being asked are not amenable to computational answers.
The goal of the visualization should be the primary goal of the client (if this is not feasible, select another of the goals). The person testing the system is not allowed to have seen the test data set before they explore it using the visualization system. They are to be given written instructions and sent off to run the test without further interaction with the system designers (the designers can have left the application running with the data set loaded, but cannot preset the visualization parameters to reveal the goal). Both the written instructions and a screen shot of the running application as the person first saw it are to be included in the final report. The client can be the person testing the system, but then the data set must not be the one they gave the team (unless they haven't seen it before). Include the person's responses, and a discussion of how well they met the goals. Include quotes from them on their reaction to how easy it was to use the system, and their suggestions for improvements.
This evaluation can be done at any point during the design process before the final in-class presentation (it can be done on a prototype design to help improve the system for the final design, or it can be done on the final design). You get more points if they are able to achieve the goals; you also get more points if you are able to incorporate their suggestions into the final implementation.
Informal client evaluation: Show the person who submitted the data set the visualization and get their feedback. What does the visualization let them see that they did not see before? How does the visualization compare to their existing techniques? You might want to include a picture of the scientist using the system. If your project was the visualization design challenge, then the instructor will be your client.
Within a team: It is expected that there will be a lot of interaction and discussion between the team members; each should freely provide input to all parts of the project. This process works best when there is engagement and interest from all participants. It works best when there are a number of iterations in the system development itself (development of prototype, evaluation, ideas for improvements). It may be fruitful to have sub-teams or individuals providing competing designs for each question, with the full team meeting to discuss which is the best. Team members should freely perform their own evaluation as the system evolves (separate from the final formal evaluation process). The goal is for you to work together to produce a useful visualization system for the data and task at hand. In the context of this course, everyone has had training on the appropriate design for the visualization system and so should participate heavily in the design.
This exercise is primarily a team effort, and it will be graded accordingly. An overall grade will be assigned for the project. Individual grades will then be assigned; these grades will be based 90% on the common grade and 10% on the teammate evaluations.
Between teams, use of external resources: The way progress is made in academic disciplines is for new teams to build on the work of earlier teams, giving credit to the source. It is expected that teams will make use of all available externally-developed libraries and toolkits to help implement the visualizations. Homeworks assignments from earlier in the class are fair game as well. It is not an honor code violation in this project to use any outside help, so long as proper credit is given in the acknowledgments or references section of the document. That said, you get extra points for succeeding at a project that requires more of a stretch beyond what is available from existing resources. You'll get fewer points for "We used team A's visualization system and formal analysis results (as presented by them just a minute ago) and applied it to our client's data set." Two different teams can get full points for applying different techniques to the same type of data and having each of their techniques meet the differing goals of the clients (they would not be in competition with each other any more than with teams doing different data sets).
Each team should develop a contract between its members that specifies who is going to be responsible for each portion of the work going forward. This document need not be turned in to the instructor; it is intended to set expectations among team members that will later be used for the members to evaluate each other. 10% of the project score for each person will be determined by the scores assigned by their team members; this document is meant to clarify what criteria will be used in this evaluation. The document may of course be changed during the course of the semester with agreement of all participants.
Each team will be different, with different skill sets and interests, so each contract will be different. All team members must participate in the design of the ideal visualization. Some things that one person might work on more than the others include: organizing and leading the team, evaluation design, running the evaluation, in-class presentation, writing up the final report, and implementing the visualization. The team should determine up front what split gives everyone responsibility for about equal portions of the work.
Remember, 90% of the project grade is common and based on the turned-in results, so everyone is should verify the quality of all work turned in. This contract is a way to set expectations for the 10% team grade at the end.
I have been asked each year to provide an example of a high-quality final report. I hesitate to do so because there are a variety of ways to present things that have succeeded, and the relative amount of implementation work and design work varies between projects. However, I include an example of a final project report that received an H here.