We hope you'll join us for our 4/23 webinar on using data tables to apply reference ranges and AE codes in OC4. For more information and to register, visit https://register.gotowebinar.com/register/2882170018956684555

SAS export (thx Linas) to be included in 3.10! Looking for community feedback

Hi Everyone,

A while back @Linas contributed code for a SAS export. We're currently working on bringing it into OpenClinica so that it will be part of the product going forward. We know that some folks have tried it out and would love to hear any feedback about how it has been working for them? @lindsay.stevens , thanks for hosting it - any feedback from you?

Really looking forward to getting this contribution out to everyone!

Best,
Alicia

«13

Comments

  • lindsay.stevenslindsay.stevens Posts: 404 ✭✭✭
    via Email
    Well I wrote a singular XSL version, just pushed it up to Github. Only
    thing I'm not sure about is whether the OC extract procedures work with
    multiple result-documents, haven't tried. Works OK with Saxon as a post hoc
    script. I also recently updated the dnotes script in case you want to
    consider that too.

    Do you guys have a test ODM XML I can use to test with? It would be
    tremendously helpful in handling all the edge cases in study design. Same
    goes for the Postgres database, I can only be sure datamart works with our
    design styles. It would be hugely helpful to have access to common test
    fixtures, even if we can't have the test code.
  • ccollinsccollins Posts: 379 admin
    via Email
    Hi Lindsay,

    For example XML, the Juno study from the demo server (
    https://www.openclinica.com/home/openclinica-demos/) is a decent place to
    start.

    The three-file SAS export XSLs work pretty well as an OpenClinica extract
    format from what we can tell so far. The extract.properties can be
    configured to handle multiple XSLs and output files. I don't remember
    offhand though if a single XSL can produce multiple output files from
    within OpenClinica. Let me know if you get a chance to test it.

    Best,
    Cal
  • tkhajatkhaja Posts: 61 ✭✭
    edited February 2016
    Hi Lindsay,

    I am currently trying to configure the SAS extract format using the singular XSL version in OpenClinica version 3.9. This is what I did so far and below is the error I am getting.

    Steps followed:
    1. Added the oc_odm_xml_to_sas.xsl(downloaded from github) to the openclinica.data/xslt/ folder
    2. Edited the extract.properties file to include this as an extract format

    Good news is tomcat is starting and I can login and view the dataset format and I am able to click run now and download the extract in SAS, however, the zip file is of size 0 bytes once downloaded. The following is the error message I am getting:

    Error Message in catalina.out file:
    Recoverable error on line 155 of oc_odm_xml_to_sas.xsl:
    FODC0005: java.net.MalformedURLException: no protocol:
    Recoverable error on line 214 of oc_odm_xml_to_sas.xsl:
    FODC0005: java.net.MalformedURLException: no protocol:

    Here are the lines on which it's failing(copied from oc_odm_xml_to_sas.xsl file)


    Let me know if you need more information.

    Regards,
    Thasbiha


  • lindsay.stevenslindsay.stevens Posts: 404 ✭✭✭
    via Email
    Yes I was looking at it briefly today and got the same error. I think it is
    because the XSLT job uses a fileoutputstream rather than a regular file,
    which means xsl result document doesn't work. [1]. Haven't tried patching
    this yet, looks like it at [2]. It should be that only one XSL and one
    output is specified in extract.properties, and the XSL will take care of
    emitting the other 2 result docs. It just needs a valid URI to work with.

    I am eager to get this working because it could also be used with Linas'
    CSV exports, which could be emitted as result documents instead of being
    chopped up after export.

    Another problem was with the self referential document trick, but I changed
    it to use a XSL variable instead, just pushed the change up to github a
    moment ago.

    Thanks for looking in to this.

    [1] https://saxonica.plan.io/boards/3/topics/2758
    [2]
    https://github.com/OpenClinica/OpenClinica/blob/b1a4ff2dd1193c55dd477e2ae78964cc49c29f8f/core/src/main/java/org/akaza/openclinica/job/XsltTransformJob.java#L241
  • Hi all,

    We have recently adapted the work by Linas (thanks, Linas and Lindsay!) to fit our needs and successfully managed to transform a standard 'CDISC ODM XML 1.3 Clinical Data with OpenClinica extensions' export to something that we can read with SAS, albeit still on the now rather antique version 3.1.3.1. We are calling the scripts manually still, as we are on Enterprise Edition and haven't yet seen the need to automate the process in another manner.

    If there's anything we can help with, let me know.

    Best, Henrik
  • tkhajatkhaja Posts: 61 ✭✭
    edited February 2016
    Hi Lindsay,

    Thank you for your response. I was able to run it as a batch file after downloading the "CDISC ODM XML 1.3 Full with OpenClinica extensions" extract. It's nice to see the study name in the output which reflects the OpenClinica study name and I also noticed that you incorporated the code to import the data and the map files as a part of the sas script. However, I am getting errors while running the load.sas script in SAS studio. I am attaching the first 2 pages of the log here.

    The following errors were repeated most times:
    • ERROR 22-322: Syntax error, expecting one of the following: ), DEFAULT, FUZZ, MAX, MIN, MULTILABEL, NOTSORTED, ROUND.

    • ERROR 22-322: Syntax error, expecting one of the following: a quoted string, a numeric constant, a datetime constant,

    • ERROR: Physical file does not exist, /pbr/biconfig/940/Lev1/SASApp/map.xml

    • ERROR 22-322: Syntax error, expecting one of the following: (, ',', -,
    Please note I didn't have these errors when I ran the format.sas script in sas studio generated from Linas script(using 3 xsl files) which we configured as one of the extract format in OpenClinica. Please note I used the same dataset in my test.

    Let me know if you need more information.

    Thanks,
    Thasbiha
  • jamesbonicajamesbonica Posts: 7
    Hi Lindsay,

    I tried running your implementation on an extract with a Study OID of S_55AUTOM and there was a runtime error:

    XTDE0820: Invalid element name. Invalid QName {55_AUTOM}.

    I then experimented with trying to load a library into my SAS Studio using the following:

    FileName 55_AUTOM '/home/jbonica0/SAS_DATA.xml';
    FileName map '/home/jbonica0/SAS_MAP.xml';
    libname 55_AUTOM xml xmlmap=map access=readonly ;

    and SAS returned an error for an invalid name "55_AUTOM."

    I experimented and SAS liked:
    S55AUTOM
    S_AUTOM

    and didn't like:
    S_55AUTOM
    S55_AUTOM
    S_AUTOM55
    55AUTOMA

    Are there any plans to handle Study OIDs that have numbers in them? For the test studies we create for our automated tests, our naming convention for unique protocol id is "(highest existng automated test number + 1)_Automation."

    We could change this for testing this new feature but are there OC users who have numbers in there protocol ids who might run into issues?

    Thanks!
    Jim B.
  • GerbenRienkGerbenRienk Posts: 810 ✭✭✭
    Hi Jim,
    Most definitely: numbers, dashes, underscores, you name it.
    Gerben Rienk
  • lindsay.stevenslindsay.stevens Posts: 404 ✭✭✭
    via Email
    Thanks for looking at the script. Could you guys please provide an XML
    file(s) that reproduce these issues?

    Jim's one sounds a bit more straightforward than Jamuna's but it would be
    really helpful to see what data you're running through it.

    Also, what version of SAS are you using? I've got 9.3.
  • jamesbonicajamesbonica Posts: 7
    edited February 2016
    Hi Lindsay,

    I'm using SAS Studio.

    When I tried to use the following to upload the library for my SAS extract:

    FileName 55_AUTOM '/home/jbonica0/SAS_DATA.xml';
    FileName map '/home/jbonica0/SAS_MAP.xml';
    libname 55_AUTOM xml xmlmap=map access=readonly ;

    SAS returns the following errors:

    FileName 55_AUTOM '/home/jbonica0/SAS_DATA.xml';
    ERROR: Invalid logical name.
    ERROR: Error in the FILENAME statement.
    57 FileName map '/home/jbonica0/SAS_MAP.xml';
    58 libname 55_AUTOM xml xmlmap=map access=readonly ;
    ERROR: 55 is not a valid SAS name.
    ERROR: Error in the LIBNAME statement.

    If I change the above to XX_AUTOM, then there is no problem.

    I've found these combos work:

    S55AUTOM
    S_AUTOM

    These do not:
    55AUTOMA
    S_55AUTOM
    S_AUTOM55
Sign In or Register to comment.