The USArray Ground Motion Visualization (GMV) is an IRIS video product that illustrates seismic waves traveling away from an earthquake by depicting seismometers as symbols that vary in color according to recorded amplitude. GMVs are typically the most popular IRIS product following an earthquake (e.g., ~10,000 unique views for an Oklahoma earthquake). Many instructors think dynamic visualizations offer advantages over static media, but research has indicated they can impede learning by making students process more information. We evaluated changes in student understanding of seismic waves from GMVs by collecting data from 3 different college-level settings: general student population in a psychology laboratory, students in geoscience majors courses, and a seismology research group. A 7-question multiple-choice assessment was developed in all 3 settings and then administered in the laboratory and classroom. Using a similar question before and after the GMV, we found most geoscience majors understood seismic wave concepts prior to the GMV and the GMV improved their understanding. Only about half the novices appeared to understand seismic wave concepts prior to the GMV and performance decreased after the GMV. Performance decreases were larger when students watched an alternative "tutorial" GMV developed to further illustrate what a GMV represents. Increased breadth of incorrect answer selections by novices indicated the GMV increased confusion about what happens to energy from an earthquake. Lower performance on other post-GMV questions by novices suggests the current style of GMVs are unable to teach basic seismological concepts to people without geoscience training. While web traffic indicates people's interest in GMVs, watching GMVs does not appear to translate to improved understanding of seismic waves for novices. Future development of dynamic visualizations should consider the cognitive load these learning materials impose on the learner and seek to further implement principles of multimedia instructional design that minimize cognitive processing demands.