Patchworkβ [1/4] test: add 'GnuPG' prereq to dependent 'crypto' tests

login
register
about
Submitter Pieter Praet
Date 2011-11-01 19:49:11
Message ID <1320176954-4897-2-git-send-email-pieter@praet.org>
Download mbox | patch
Permalink /patch/1444/
State New
Headers show

Comments

Pieter Praet - 2011-11-01 19:49:11
Adds a new test that checks for the presence of 'gpg',
and adds that test as a prereq to all subsequent tests
that rely on GnuPG.

This causes tests with unmet dependencies to be skipped.

---
 test/crypto |   33 +++++++++++++++++++--------------
 1 files changed, 19 insertions(+), 14 deletions(-)
Jameson Graef Rollins - 2011-11-01 21:17:51
On Tue,  1 Nov 2011 20:49:11 +0100, Pieter Praet <pieter@praet.org> wrote:
> -test_expect_success 'emacs delivery of signed message' \
> +test_expect_success GPG 'emacs delivery of signed message' \

Hi, Pieter and Thomas.  Thanks for all the work on this, but I have one
issue.  Is there a way we can do this without adding a new argument to
every test function?  For some reason I really don't like that solution.
It seems too invasive.  Can't we have something that works more like
test_subtest_known_broken, that modifies the test environment, rather
than add an argument to every call of every testing function?

jamie.
Pieter Praet - 2011-11-02 17:20:09
On Tue, 01 Nov 2011 14:17:51 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Tue,  1 Nov 2011 20:49:11 +0100, Pieter Praet <pieter@praet.org> wrote:
> > -test_expect_success 'emacs delivery of signed message' \
> > +test_expect_success GPG 'emacs delivery of signed message' \
> 
> Hi, Pieter and Thomas.  Thanks for all the work on this, but I have one
> issue.  Is there a way we can do this without adding a new argument to
> every test function?  For some reason I really don't like that solution.
> It seems too invasive.  Can't we have something that works more like
> test_subtest_known_broken, that modifies the test environment, rather
> than add an argument to every call of every testing function?
> 
> jamie.

I've been thinking the very same thing.


We could use `test_have_prereq' to get rid of the argument, e.g.:

  #+begin_src sh
    test_have_prereq EMACS && \
    test_begin_subtest "blah"
    echo "doing stuff..."
    test_expect_equal_file OUTPUT EXPECTED
  #+end_src

...but still, everything between `test_begin_subtest' and `test_expect_*'
would be executed, so to err on the safe side, we could wrap the full body
of every test, e.g.:

  #+begin_src sh
    test_begin_subtest "blah"
    test_have_prereq EMACS && {
    ...
    echo "doing stuff..."
    ...
    }
    test_expect_equal_file OUTPUT EXPECTED
  #+end_src


Or... (I've given this zero thought, so please take it with a bag of
salt) we could run all tests from inside a "controller" which `eval's
their contents line by line, and skips a test entirely whenever $? > 0.


All of this is (still) excessively invasive in some way or another though...

Suggestions?


Peace

Patch

diff --git a/test/crypto b/test/crypto
index 0af4aa8..3795926 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,6 +7,11 @@ 
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
+# GnuPG is a prereq.
+test_expect_success "prereq: GnuPG is present" "which gpg" \
+    && test_set_prereq GPG
+
+
 add_gnupg_home ()
 {
     local output
@@ -31,7 +36,7 @@  FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep
 # although I can't figure out why
 add_email_corpus
 
-test_expect_success 'emacs delivery of signed message' \
+test_expect_success GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
     "test signed message 001" \
     "This is a test signed message." \
@@ -64,7 +69,7 @@  expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -99,7 +104,7 @@  expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -132,7 +137,7 @@  expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -141,7 +146,7 @@  mv "${GNUPGHOME}"{.bak,}
 cat <<EOF >TESTATTACHMENT
 This is a test file.
 EOF
-test_expect_success 'emacs delivery of encrypted message with attachment' \
+test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
 'emacs_deliver_message \
     "test encrypted message 001" \
     "This is a test encrypted message.\n" \
@@ -175,7 +180,7 @@  Non-text part: application/octet-stream
 part}
 body}
 message}'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -210,7 +215,7 @@  expected='[[[{"id": "XXXXX",
  "content-type": "application/octet-stream",
  "filename": "TESTATTACHMENT"}]}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -221,7 +226,7 @@  output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted m
 expected='{"id": 4,
  "content-type": "text/plain",
  "content": "This is a test encrypted message.\n"}'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -231,7 +236,7 @@  notmuch show \
     --part=5 \
     --decrypt \
     subject:"test encrypted message 001" >OUTPUT
-test_expect_equal_file OUTPUT TESTATTACHMENT
+test_expect_equal_file GPG OUTPUT TESTATTACHMENT
 
 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
@@ -258,12 +263,12 @@  expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
 
-test_expect_success 'emacs delivery of encrypted + signed message' \
+test_expect_success GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
     "test encrypted message 002" \
     "This is another test encrypted message.\n" \
@@ -298,7 +303,7 @@  expected='[[[{"id": "XXXXX",
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -310,7 +315,7 @@  Subject: Re: test encrypted message 002
 
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is another test encrypted message.'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -351,7 +356,7 @@  expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"