[ << Regression tests ]	[Top][Contents][Index][ ? ]			[ Programming work >> ]
[ < Compiling regression tests ]		[  Up : Regression tests ]	[ Finding the cause of a regression > ]

9.4 Regtest comparison

Before modified code is committed to master, a regression test comparison must be completed to ensure that the changes have not caused problems with previously working code. The comparison is made automatically upon compiling the regression test suite twice.

1. Run make with current git master without any of your changes.
2. Before making changes to the code, establish a baseline for the comparison by going to the ‘lilypond-git/build/’ directory and running:

   make test-baseline
   

3. Make your changes, or apply the patch(es) to consider.
4. Compile the source with ‘make’ as usual.
5. Check for unintentional changes to the regtests:

   make check
   

   After this has finished, a regression test comparison will be available (relative to the current ‘build/’ directory) at:

   out/test-results/index.html
   

   For each regression test that differs between the baseline and the changed code, a regression test entry will be displayed. Ideally, the only changes would be the changes that you were working on. If regressions are introduced, they must be fixed before committing the code.

   Note: The special regression test ‘test-output-distance.ly’ will always show up as a regression. This test changes each time it is run, and serves to verify that the regression tests have, in fact, run.

6. If you are happy with the results, then stop now.

   If you want to continue programming, then make any additional code changes, and continue.

7. Compile the source with ‘make’ as usual.
8. To re-check files that differed between the initial ‘make test-baseline’ and your post-changes ‘make check’, run:

   make test-redo
   

   This updates the regression list at ‘out/test-results/index.html’. It does not redo ‘test-output-distance.ly’.

9. When all regressions have been resolved, the output list will be empty.
10. Once all regressions have been resolved, a final check should be completed by running:

    make test-clean
    make check
    

    This cleans the results of the previous ‘make check’, then does the automatic regression comparison again.

---

[ << Regression tests ]	[Top][Contents][Index][ ? ]			[ Programming work >> ]
[ < Compiling regression tests ]		[  Up : Regression tests ]	[ Finding the cause of a regression > ]

<< Back to Documentation Index

LilyPond — Contributor’s Guide