Possibly more rose-related; can’t (manually) see anything obvious in previous qs - discourse cleverly picks up the tangentially-related “Pass command line env vars…” q (and a few false positives), but believe still enough new stuff here to warrant its own q.
I’m wanting to have an overall integration/end-to-end “test” mode for the suite I’m designing. This to ensure things like the overall graph/intercycle dependencies etc work as intended - I’ve got some quite thorny suicide/retry/restart aspects to manage.
I’m hoping to use a “test-driven development” approach as I build this, as reckon this might help avoid tears later. Using placeholder scripts & local compute to get faster feedback than if I iteratively develop using my real applications - spending ages waiting for runs to complete, needlessly hammering shared compute resources etc!
For a given app “task-a” I’m pretty happy I can set up some dummy scripts “task-a_test_bad-data.sh”, “task-a_test_datagap.sh” which mimic the expected failure modes of the real “task-a.sh”, but not involve actual computation, so fail as fast as I want, and cheaply. So I can check the design works as-intended.
And that per-app I can use optional configurations to call these different dummy scripts.
And that I can effectively use this to create ~unit/low-level integration tests for given bits by leveraging
rose app-run --opt-conf-key
However, I’m interested in higher level integration/end-to-end test. I.e. probably only picking a few pathways through the myriad combinations of snafus with “dummy” scripts - middle tier of test pyramid; or single pathway through with the real script using canned data (top tier).
I’m not clear how I can achieve this cleanly (i.e. with as little Jinja/duplication as possible).
I think this might be using top-level rose-suite.conf suite configuration(s), which switch on the various tests.
And that this/these configurations could:
- enable various
ROSE_APP_OPT_CONF_KEYSin the apps - e.g. an end-to-end
rose-suite.confmight set something (???) in the
suite.rcwhich would (where applicable) switch on the configs suitable for an end-to-end test in all apps
- let me specify alternative hosts for tasks (e.g. just local host), so I can avoid hitting shared/expensive compute hosts
And that overall, I might (haven given this much thought) even be able to conveniently expose my desired test harness level (all unit/all integration/end-to-end) on the command line, allowing usage like
rose suite-run - S 'TEST=end-to-end', using @oliver.sanders’ point in the Pass command line env vars q.
However, grateful of any pointers here as to what best approach would be (including “you’re thinking about this incorrectly”!).
Especially keen if this can recommend an approach which avoids too much Jinja / duplication!