| [ << Regression tests ] | [Top][Contents][Index][ ? ] | [ Programming work >> ] | ||
| [ < Introduction to regression tests ] | [ Up : Regression tests ] | [ Compiling regression tests > ] | ||
9.2 Precompiled regression tests
Regression test output
As part of the release process, the regression tests are run for every LilyPond release. Full regression test output is available for every stable version and the most recent development version.
Regression test output is available in HTML and PDF format. Links to the regression test output are available at the developer’s resources page for the version of interest.
The latest stable version of the regtests is found at:
http://lilypond.org/doc/stable/input/regression/collated-files.html
The latest development version of the regtests is found at:
http://lilypond.org/doc/latest/input/regression/collated-files.html
Regression test comparison
Each time a new version is released, the regtests are compiled and the output is automatically compared with the output of the previous release. The result of these comparisons is archived online:
http://lilypond.org/test/
Checking these pages is a very important task for the LilyPond project. You are invited to report anything that looks broken, or any case where the output quality is not on par with the previous release, as described in Bug reports.
Note: The special regression test ‘test-output-distance.ly’ will always show up as a regression. This test changes each time it is run, and serves to verify that the regression tests have, in fact, run.
What to look for
The test comparison shows all of the changes that occurred between the current release and the prior release. Each test that has a significant difference in output is displayed, with the old version on the left and the new version on the right.
Regression tests whose output is the same for both versions are not shown in the test comparison.
-
Images: green blurs in the new version show the approximate
location of elements in the old version.
There are often minor adjustments in spacing which do not indicate any problem.
-
Log files: show the difference in command-line output.
The main thing to examine are any changes in page counts – if a file used to fit on 1 page but now requires 4 or 5 pages, something is suspicious!
- Profile files: give information about TODO? I don’t know what they’re for.
Note: The automatic comparison of the regtests checks the LilyPond bounding boxes. This means that Ghostscript changes and changes in lyrics or text are not found.
| [ << Regression tests ] | [Top][Contents][Index][ ? ] | [ Programming work >> ] | ||
| [ < Introduction to regression tests ] | [ Up : Regression tests ] | [ Compiling regression tests > ] | ||