Message Signing And Verification
From OMFWiki
Contents |
Background
In the Observatory Middleware Framework project, users and instruments will communicate via JMS message queues routed by an ESB. While JMS provides many nice features (such as reliable delivery of messages), there is the issue of access protection for JMS queues. Basically, any entity can deliver a message to a given JMS queue. While ActiveMQ provides some security with respect to authentication and authorization, we do not want to rely on this protocol-specific security mechanism. Instead, we will sign and verify messages using X.509 credentials.
Every entity in our system (instruments, users, ESBs, etc.) that wishes to send and/or receive messages will be given an X.509 credential. This credential can be utilized to sign messages. "End entities" such as instruments and users will send messages only to the locally connected ESB. The ESB will accept any message with a valid signature. The ESB will then resign the message with its credential and forward the message to the recipient end entity. The recipient will not only verify that the message has a valid signature, but also make sure that the message was signed with the local ESB's credential. This prevents a malicious user from sending messages to end entities.
Message signing is accomplished using the WSS4J (Web Services Security For Java) toolkit. Messages that can be signed are SOAP messages. Basically, the syntax of a SOAP message is simply an XML string in a specific format. A non-SOAP XML string can be "soapified" by putting the XML string into the body of the soap message. For example, here is a very basic XML string.
<?xml version="1.0"?>
<books>
<book>
<title>War and Peace</title>
<author>Leo Tolstoy</author>
</book>
</books>
This can be transformed into the following SOAP message.
<?xml version="1.0" encoding="UTF-16"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <books> <book> <title>War and Peace</title> <author>Leo Tolstoy</author> </book> </books> </SOAP-ENV:Body> </SOAP-ENV:Envelope>
This SOAP message can be signed by a credential and sent to another entity for verification.
Preliminaries
In order to run the examples given below, you will first need to download and install the following software.
- Java Development Kit version 1.5 (aka Java 5) or higher.
- Apache Ant version 1.6 or higher
- Subversion client (svn)
Installation and configuration details can be found on the respective project pages. At the very least, you will probably need to set the $JAVA_HOME and $ANT_HOME environment variables, and make sure that the java, javac, ant, and svn binaries are in your path.
Download / Build The Code
Change to a directory where you have write access. Check out the "Soap Signer" library code with the following command.
svn co https://svn.ncsa.uiuc.edu/svn/omf/trunk/soapsigner
This will create a new directory soapsigner containing the source code. Change to this directory and build the code.
cd soapsigner ant
You can also run some basic tests (which require no configuration).
ant run
If you were able to successfully build the code, you will have a new file release/SoapSigner.jar. By adding that jar (along with the jars in the lib subdirectory) to your CLASSPATH, you can use the library methods to sign and verify XML strings.
You can also generate the Javadoc API documentation as follows.
ant javadoc
Then open the file doc/api/index.html in your favorite browser.
X.509 Credentials
In order to sign XML strings, you need a credential. A "credential" consists of a public certificate and a private key. You probably already have an X.509 credential. However, this credential needs to be stored in a Java keystore in order to be utilized by Java code.
(Optional) Create A Self-Signed Credential
(Skip this section if you already have an X.509 credential.) In case you don't have an X.509 credential, you can use OpenSSL to generate a self-signed certificate from scratch along with a corresponding private key.
# openssl req -x509 -days 365 -newkey rsa:1024 -keyout selfkey.pem -nodes -out selfcert.pem Generating a 1024 bit RSA private key ...++++++ ........++++++ writing new private key to 'selfkey.pem' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [GB]:US State or Province Name (full name) [Berkshire]:Illinois Locality Name (eg, city) [Newbury]:Urbana Organization Name (eg, company) [My Company Ltd]:Test Company Organizational Unit Name (eg, section) []:Testing Common Name (eg, your name or your server's hostname) []:Joe Smith Email Address []:jsmith@gmail.com
You now have a self-signed public certificate in selfcert.pem and the corresponding private key in selfkey.pem.
Convert PEM Certificate To PKCS12
In order to import your credential into a Java keystore, the credential needs to be transformed from the two-file PEM format to a single-file PKCS12 format. This can be accomplished with OpenSSL.
# openssl pkcs12 -export -in selfcert.pem -inkey selfkey.pem \
-out selfcred.p12 -passout pass:temppass
Here the two separate files containing the public certificate and the private key are combined into a single password-protected credential file. Note that the password temppass can be changed if you like. However, the .p12 file is used only for importing into the Java keystore and can be deleted afterwards, so the password choice is not very important. If you prefer to keep the .p12 file, you can omit the "-passout ..." command line option from the openssl command above and you will be prompted to type in a (non-echoed) password.
Import PKCS12 Credential Into Java Keystore
In order for Java code to use your credential, it must be imported into a Java keystore. To do this, you will need to use the keytool command. This program is provided as part of the Java distribution.
keytool -importkeystore \
-srckeystore selfcred.p12 \
-srcstorepass temppass \
-srcstoretype pkcs12 \
-srcalias `keytool -list -v -keystore selfcred.p12 \
-storetype pkcs12 -storepass temppass | \
egrep '^Alias name:' | sed 's,Alias name: ,,g'` \
-destkeystore my.keystore \
-deststorepass goodpass \
-deststoretype jks \
-destalias mykey \
-destkeypass goodpass
Note that the -deststorepass and -destkeypass passwords do not need to be the same. You will probably want to choose good passwords since the newly created my.keystore file will be used to sign XML strings.
You can verify the contents of the resulting my.keystore file with this command.
keytool -list -v -keystore my.keystore -storepass goodpass
Trust Other Certificates
At this point, you have a keystore with which you can sign XML strings. You can also use this same keystore to verify previously signed XML strings. (Actually, the contents of the signed message should be sufficient for verification. However, all WSS4J library methods require a valid keystore to be read in.)
Sometimes, you will want to go one step further and trust the signed message. Basically, you want to verify the signed message AND make sure it comes from a predefined set of trusted signers. In our case, we want instruments to accept messages only from the local ESB. To do this, you must import another signer's public certificate (e.g. from the local ESB) into your previously created keystore. You will need the other signer's public certificate .pem file. Alternatively, this certificate can be exported from the signer's keystore as follows.
# keytool -exportcert -rfc -keystore esb.keystore -storepass esbgoodpass \
-alias mykey -file pubcert.pem
Certificate stored in file <pubcert.pem>
The pubcert.pem file can be imported into the local keystore (which is now also being used as a truststore) as follows.
# keytool -importcert -keystore my.keystore -storepass goodpass \
-alias trustesb -file pubcert.pem
The "-alias ..." can be anything you like, but must be unique for each public certificate you want to trust. In our case, an instrument will only trust the local ESB to which it is connected, so you will need to import only one public certificate into the keystore.
Java Example Code
Now that the Soap Signer library has been built and the keystore created, we can write some code to actually use them. The examples below can be typed in using your favorite editor. Make sure you are in the soapsigner directory.
Set The Java Classpath
Both the javac and java commands require that the classpath be set to include the libraries you need. All of the .jar libraries reside in the lib and release subdirectories. To simplify the commands, we can set the $CLASSPATH environment variable as follows (on Linux).
# export CLASSPATH="."`find . -name *.jar -printf ":%h/%f"`
If you are using Java 1.6, you can use the easier wildcard method.
# export CLASSPATH=.:lib/*:release/*
Now we don't need to use the "-classpath ..." command line option when building or running the examples.
Skeleton Test Code
In the soapsigner directory, create a new file named "SignAndVerify.java" and enter the following skeleton Java program.
import edu.uiuc.ncsa.soap.SOAPUtil;
import edu.uiuc.ncsa.soap.SOAPSign;
import org.w3c.dom.Document;
public class SignAndVerify {
public static final String xmlString =
"<?xml version=\"1.0\"?>\n" +
"<books>\n" +
" <book>\n" +
" <title>War and Peace</title>\n" +
" <author>Leo Tolstoy</author>\n" +
" </book>\n" +
"</books>";
public static void main(String[] args) {
String soapString = "";
try {
Document xmlDoc = SOAPUtil.xmlStringToDocument(xmlString);
Document soapDoc = SOAPUtil.soapifyDocument(xmlDoc);
soapString = SOAPUtil.documentToString(soapDoc);
System.out.println(soapString);
} catch (Exception e) {
e.printStackTrace();
}
}
}
Save the file. Then compile and run the program as follows.
# javac SignAndVerify.java # java SignAndVerify <?xml version="1.0" encoding="UTF-16"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"> <SOAP-ENV:Header/><SOAP-ENV:Body><books> <book> <title>War and Peace</title> <author>Leo Tolstoy</author> </book> </books></SOAP-ENV:Body></SOAP-ENV:Envelope>
Basically, this takes an XML string and turns it into a SOAP-formatted XML string, which is printed to stdout. Note that the SOAPUtil class also has a method to read in an XML file. So if the xmlString was saved in the file "test.xml", you could change the first line to the following.
Document xmlDoc = SOAPUtil.xmlFileToDocument("test.xml");
Sign The XML String
When signing a message, the WSS4J library uses the concept of Crypto Properties. These properties include the name and type of the keystore, the password for the keystore and private alias/key, etc. These properties can be read from a file or set programmatically.
Read crypto.properties File
If you want to store your Crypto properties in a file, create a new file named "crypto.properties" with your favorite editor and enter the following.
org.apache.ws.security.crypto.provider=org.apache.ws.security.components.crypto.Merlin org.apache.ws.security.crypto.merlin.keystore.type=jks org.apache.ws.security.crypto.merlin.file=my.keystore org.apache.ws.security.crypto.merlin.keystore.password=goodpass org.apache.ws.security.crypto.merlin.keystore.alias=mykey org.apache.ws.security.crypto.merlin.alias.password=goodpass
To use this file for message signing, modify the skeleton program as follows. (Added/modified code is in bold.)
...
public static void main(String[] args) {
String soapString = "";
try {
Document xmlDoc = SOAPUtil.xmlStringToDocument(xmlString);
Document soapDoc = SOAPUtil.soapifyDocument(xmlDoc);
SOAPSign soapSign = new SOAPSign();
soapSign.getCryptoProperties().loadFile("crypto.properties");
Document signDoc = soapSign.signDocument(soapDoc);
soapString = SOAPUtil.documentToString(signDoc);
System.out.println(soapString);
} catch (Exception e) {
e.printStackTrace();
}
}
...
Now compile and run the program as before.
# javac SignAndVerify.java # java SignAndVerify <?xml version="1.0" encoding="UTF-16"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header> <wsse:Security SOAP-ENV:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" wsu:Id="CertId-7338350" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">MIIDljCCAv+gAwIBAgIJAO8z1K0qreHEMA0GCSqGSIb3DQEBBQUAMIGPMQswCQYDVQQGEwJVUzERMA8GA1UECBMISWxsaW5vaXMxDzANBgNVBAcTBlVyYmFuYTEVMBMGA1UEChMMVGVzdCBDb21wYW55MRAwDgYDVQQLEwdUZXN0aW5nMRIwEAYDVQQDEwlKb2UgU21pdGgxHzAdBgkqhkiG9w0BCQEWEGpzbWl0aEBnbWFpbC5jb20wHhcNMDgwOTI1MTgyOTEyWhcNMDkwOTI1MTgyOTEyWjCBjzELMAkGA1UEBhMCVVMxETAPBgNVBAgTCElsbGlub2lzMQ8wDQYDVQQHEwZVcmJhbmExFTATBgNVBAoTDFRlc3QgQ29tcGFueTEQMA4GA1UECxMHVGVzdGluZzESMBAGA1UEAxMJSm9lIFNtaXRoMR8wHQYJKoZIhvcNAQkBFhBqc21pdGhAZ21haWwuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCXMA0IwMO8SIvj9+MYmVS+oVHfgCxdvmvdrXeMPAGSs25DAq4LEfWm7hN94tbg0H5tSHGTGfgOExDOtcJEcYMr7pZa2dHz731mNfw0JprQwvGU+CGWbi3ulbrrYs68ofxVHwxf2D8T4KoR3DTWKcbXv/2eQuXEDFxH8ip3TuDmhwIDAQABo4H3MIH0MB0GA1UdDgQWBBTo0a1iRTPLMM73cedODKdoyOS5kTCBxAYDVR0jBIG8MIG5gBTo0a1iRTPLMM73cedODKdoyOS5kaGBlaSBkjCBjzELMAkGA1UEBhMCVVMxETAPBgNVBAgTCElsbGlub2lzMQ8wDQYDVQQHEwZVcmJhbmExFTATBgNVBAoTDFRlc3QgQ29tcGFueTEQMA4GA1UECxMHVGVzdGluZzESMBAGA1UEAxMJSm9lIFNtaXRoMR8wHQYJKoZIhvcNAQkBFhBqc21pdGhAZ21haWwuY29tggkA7zPUrSqt4cQwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOBgQB89FRd4LR61urpwlazICH5eSPZQjwzDz9/Fp2K6XbB9XM6fNbDFLPQFyE5rgxuXyIwlqo1eYH1JqlCmKBg0EBEwYYaDjsRRUxgQsqMAU+LgHtPRYCeBzR4kaYOYAXwXHHMvoXltfWMV1mqGe+qy2zEVfH7i1CidmAHSqmWG3TapQ==</wsse:BinarySecurityToken><ds:Signature Id="Signature-30911772" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> <ds:SignedInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"/> <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"/> <ds:Reference URI="#id-9532399" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> <ds:Transforms xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"/> </ds:Transforms> <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"/> <ds:DigestValue xmlns:ds="http://www.w3.org/2000/09/xmldsig#">xILokMq8D4KtLnEIN+vIYjyODuE=</ds:DigestValue> </ds:Reference> </ds:SignedInfo> <ds:SignatureValue xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> PppEple2vff5V4YDtHFORguXpL4LxFI7uGbYeB3JG/ySGCjkwAbYuvCZUKvLZm+tELSP32wgbaln bBQx7Z1SDiQo4T1tPh+qIfYTee/nFtU0CiLr79VezeIGEp3FfH96GEB2T0Byq/278b1E5MQrvsBS Aksbx3J6r2jVE3ECEY4= </ds:SignatureValue> <ds:KeyInfo Id="KeyId-12568800" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> <wsse:SecurityTokenReference wsu:Id="STRId-26204548" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsse:Reference URI="#CertId-7338350" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"/></wsse:SecurityTokenReference> </ds:KeyInfo> </ds:Signature></wsse:Security></SOAP-ENV:Header><SOAP-ENV:Body wsu:Id="id-9532399" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><books> <book> <title>War and Peace</title> <author>Leo Tolstoy</author> </book> </books></SOAP-ENV:Body></SOAP-ENV:Envelope>
The two important entries in the output are highlighted in green. You will need to scroll the screen to the right to see the first very long string identified by the wsse:BinarySecurityToken tag. This is the base64 representation of the public certificate utilized to sign the messge. It will be used to make sure that the message came from a "trusted" sender. The second string identified by the ds:SignatureValue tag is the actual signature calculated for the message body. It will be used to verify that the message body contents were not altered during transmission.
Set Crypto Properties Programmatically
If you prefer, you can set all of the values from the crypto.properties file programmatically (i.e. at runtime). Below is the code setting values manually.
import edu.uiuc.ncsa.soap.CryptoProperties;
...
public static void main(String[] args) {
String soapString = "";
try {
Document xmlDoc = SOAPUtil.xmlStringToDocument(xmlString);
Document soapDoc = SOAPUtil.soapifyDocument(xmlDoc);
SOAPSign soapSign = new SOAPSign();
CryptoProperties cryptoProps = new CryptoProperties();
cryptoProps.setKeystoreFilename("my.keystore");
cryptoProps.setKeystorePassword("goodpass");
cryptoProps.setKeystoreAlias("mykey");
cryptoProps.setAliasPassword("goodpass");
soapSign.setCryptoProperties(cryptoProps);
Document signDoc = soapSign.signDocument(soapDoc);
soapString = SOAPUtil.documentToString(signDoc);
System.out.println(soapString);
} catch (Exception e) {
e.printStackTrace();
}
}
...
Note that we do not need to set the crypto.provider or the keystore.type since those are default values set in the CryptoProperties class upon new object creation. Now, if you compile and run this example as you did before, you should get the same output.
Verify The Signed XML String
Now that we have a signed XML/SOAP string, we want to verify the signature and (possibly) make sure that the message was signed by a trusted certificate. If the document signature is verified, we then want to extract out the original XML message string. So continuing with the SignAndVerify.java example code, add new code to the bottom of the main method.
...
soapString = SOAPUtil.documentToString(signDoc);
// System.out.println(soapString);
} catch (Exception e) {
e.printStackTrace();
}
try {
Document signDoc = SOAPUtil.xmlStringToDocument(soapString);
SOAPSign soapSign = new SOAPSign();
soapSign.getCryptoProperties().loadFile("crypto.properties");
boolean trusted = soapSign.trustDocument(signDoc);
System.out.println("Signature verified. Trusted = " + trusted);
Document xmlDoc = SOAPUtil.deSoapifyDocument(signDoc);
String origString = SOAPUtil.documentToString(xmlDoc);
System.out.println(origString);
} catch (Exception e) {
System.out.println("Signature NOT verified.");
e.printStackTrace();
}
...
Due to the way the WSS4J library is structured, the only way to find out if the signature cannot be verified is to catch exceptions thrown by the trustDocument() method. If that method does not throw an exception, then the document signature is valid. The method then checks the keystore for a public certificate that matches the one utilized to sign the document and, if found, returns true for "trusted".
Now compile and run the program to verify the signed message string.
# javac SignAndVerify.java
# java SignAndVerify
Signature verified. Trusted = true
<?xml version="1.0" encoding="UTF-16"?>
<books>
<book>
<title>War and Peace</title>
<author>Leo Tolstoy</author>
</book>
</books>
Note that when you run the example verify/trust code above, you are using the same keystore as you used to sign the document. Thus it is trusted since the public certificate embedded in the signed XML string matches the public certificate in the keystore. If you want to do a more rigorous test, you will need to sign the document with a different X.509 credential / keystore.
As before, you can set the Crypto properties programmatically rather than read them from a file.
...
soapString = SOAPUtil.documentToString(signDoc);
// System.out.println(soapString);
} catch (Exception e) {
e.printStackTrace();
}
try {
Document signDoc = SOAPUtil.xmlStringToDocument(soapString);
SOAPSign soapSign = new SOAPSign();
CryptoProperties cryptoProps = new CryptoProperties();
cryptoProps.setKeystoreFilename("my.keystore");
cryptoProps.setKeystorePassword("goodpass");
soapSign.setCryptoProperties(cryptoProps);
boolean trusted = soapSign.trustDocument(signDoc);
System.out.println("Signature verified. Trusted = " + trusted);
Document xmlDoc = SOAPUtil.deSoapifyDocument(signDoc);
String origString = SOAPUtil.documentToString(xmlDoc);
System.out.println(origString);
} catch (Exception e) {
System.out.println("Signature NOT verified.");
e.printStackTrace();
}
...
Notice here that we do not need to set values for keystore.alias or alias.password. (These values could also be removed from the crypto.properties file for signature verification.) When we signed the document, we needed access to a private key (alias), thus we needed to specify the key alias we wanted to use as well as the password needed to access it. When we verify/trust a signature, we only access the public certificates of the keystore. Thus we do not need to specify a particular key alias or an alias password.
Illustrations
It has been said that a picture is worth a thousand words. Below are several conceptual illustrations of the above discussion. Note that while the Soap Signer library operates on org.w3c.dom.Document objects internally, the details of converting from XML string to Document and back are omitted from the diagrams.
Import X.509 Credential Into Keystore
The first step is to transform an existing X.509 credential (stored as separate files for the public certficate and the private key) into a Java keystore. Also, this image shows importing another public certificate (from the ESB for example) for "trusting" a signed message.
Sign An XML String
Here we take an XML string, "soapify" it by wrapping it in a SOAPEnvelope and SOAPBody, and sign the resulting SOAP message using the private key in the keystore. Note that the public certificate corresponding to the private key is also put in the resulting signed message for later verification.
Verify A Signed XML String
Finally, the signed SOAP message needs to be verified by the recipient. The trustDocument() method does double duty. It first uses the public certificate in the SOAP message to verify the digital signature. Then it tries to find the public certificate in the local keystore for validation of trust. If successful, the SOAP Body is extracted to get the original XML string.
