Literature Survey & References

Key Conferences

• VISSOFT
  • VISSOFT 2009 2009_TOC
  • VISSOFT 2007 2007_TOC
  • 2005_TOC
• SoftVis
  • 2008 TOC
  • 2006 TOC
  • 2005_TOC
  • 2003 TOC
• VisWeek 2010
• BELIV workshops (hosted in CHI)
  • 2008_TOC
  • 2006_TOC
• IEEE VIS and INFOVIS 2009 TOC
• IEEE ICPC (International Conference on Program Comprehension)
  • ICPC_2009.pdf
  • ICPC_2008.pdf

There are several key conferences, including: ACM Symposium on Software Visualization (SoftVis) 2003, 2005, 2006; IEEE International Workshop on Visualizing Software for Understanding and Analysis; ICSE Workshop on Software Visualization, 2001; OOPSLA Workshop on SoftwareVisualisation, 2001. A few other conferences also have relatively high concentrations of software visualisation papers, including: International Workshop on Program Comprehension (IWPC) and Visual Languages/Human-Centric Computing (VL/HCC), which has taken various names over the years. In addition, there are conferences on program visualisation, such as the biennial Program Visualization Workshops, 2000, 2002, 2004, and Visualisation of Software, reported in an April 2001 special issue of Informatik.

Writing good InfoVis papers

source: 2007 InfoVis workshop report (Stasko et al)

• distinction between papers that attempt to prove the value of a visualization and those that attempt to inform future design
• clear description of the aim of the evaluation or the research question, why the selected evaluation method was chosen to answer the question, and complete description of the method, results, and lessons learned.

Problems of SoftVis

• SV tools are too domain specific - why this is a problem? (predetermined representataions, Stasko's "A knowledge task-based framework" )
• End-user visualisation: SoftVis tools are used by software developers. Some kinds of visualisation may be helpful for end-users who want to get some information about an application they are using. Diehl's SoftVis book p166
• SoftVis research agenda 1993
  • scope: Most SV systems are still dealing primarily with toy programs; many issues of scalability remain to be solved
  • The content of an SV display can vary widely, important to know SV systems intended goal and select content accordingly, more wotk to be done in transition between different type of content
  • form: interfaces for controlling multiple views to improve, ability to change levels of granularity of view, synchronized view
  • methods to specify visualisation is crude, more automation should be provided
  • effectiveness not many SV systems are evaluated

InfoVis VS SoftVis VS Visual Analytics

• "Software visualisation is the use of the crafts of typography, graphic design, animation and cinematography with modern human-computer interaction abd computer graphics technology to facilitate both the human understanding and effective use of computer software." (Price)

Object-Oriented Software Metrics

• Analyzing Java software by combining metrics and program visualization
  • "If the LCOM measure of a class is high, but the RFC and CBO values are low, then it can be suspected that the class might have unused variables or the variables have not been properly selected for the class", "CC and WMC can help recognize complex data structure", "NOC and DIT can help estimate the reusability and extensibility"
• A validation of Object-Oriented Design Metrics as Quality Indicators (1996)
  • hypotheses on the 6 metrics

InfoVis/SoftVis/VA Evaluation

Insight-based Evaluation

• "Toward Measuring Visualization Insight" 2006 g68
  • presented cons of controlled experiments on benchmark tasks (e.g. tasks are predefined, results are limited to only the tasks chosen by evaluators)

Others

• A task-oriented evaluation of five dynamic visualization tools a comparative evaluation of of dynamic visualisation tools

• Evaluating Visual Analytics System for Investigative Analysis: Deriving Design Principles from a Case Study
  • not evaluating Jigsaw per se, but evaluate how visualization can assist investigative analysis

• Beyond Usability: Evaluation Aspects of Visual Analytic Environments
  • proposed 5 key aspects for VA tools/environments evaluation: situation awareness, collaboration, interaction, creativity, and utility

• A Knowledge Task-Based Framework for Design and Evaluation of Information Visualizations
  • discuss the notion of analytic gaps, which represent obstacles faced by visualizations in facilitating higher-level analytic tasks, such as decision-making and learning.
  • rationale gap: the gap between seeing a relationship and being able to explain the nature of the relationship
  • world view gap: the difference between what is shown in the visualization and the visualization that would be needed in order to be directly used in a decision-making process
  • use the knowledge sub-tasks as a form of heuristic evaluation (section 5.2)

• Heuristics for Information Visualisation Evaluation (2006)
  • a preliminary exploration of how we might develop a set of appropriate heuristics for evaluation of InfoVis
  • heuristics taxonomy: perception (Zuk and Carpendale's), usability (Schneiderman's), discovery process (Amar and Stasko's)

• The Challenge of Information Visualization Evaluation
  • a literature survey of 50 user studies, fond 4 thematic areas of evaluation
    • Controlled experiment comparing design elements
    • Usability evaluation of a tool
    • Controlled experiments comparing two or more tools
    • Case studies of tools in realistic settings

SoftVis literature

• Various visualisation tools are available on the web, on individual websites, and in compilations such as %u2018SCG Smallwiki CodeCrawler: A non-exhaustive list of software visualisation tools%u2019.

• Four anthologies give good overviews: Software Visualization (Eades and Zhang, 1996); Software Visualization: Programming as a Multimedia Experience (Stasko et al., 1998), which includes an authoritative %u2018Early History%u2019 chapter by Baecker and Price; Software Visualization (Diehl, 2002), which resulted from a Software Visualization Dagstuhl (i.e., day symposium) in 2001; and Software Visualization: Theory into Practice (Zhang, 2003), a collection of extended versions of refereed papers from a special issue of the cancelled Annals of Software Engineering. The first three encompass both software visualisation and program visualisation, the fourth focuses on software visualisation.

• Software Visualization in the Large (Ball & Eick, 1996)
  • "It is tedious to reconstruct complex system behavior by analyzing code."

• Comprehension with[in] Virtual Environment Visualisations (Knight & Munro 1999)
  • SoftVis definition and origin, motivation
  • "It has long been known that understanding software is a complex and hard task because of the complexity of the software itself."

Visual Software Analytics

• A Visual Analytics Tool for Software Project Structure and Relationships among Classes

Other references

Craig's SoftVis references collection

John Stasko Publications

Bibliography of a InfoVis course taught in gatech by John Stasko

Background Literature on Evaluation for Information Visualization

Chris North's publications