31 Dec 2018: Thank you to our knowledgeable and friendly forums users for another great year. We are aware of the uptick in spam accounts and are doing our best to ban these at first sight. Thank you for your patience!

Need urgent help on large dataset extraction

RCHENURCHENU Posts: 203
edited August 2016 in Extracting & Reporting
Hi,

I try to create a dataset but it's not working:

1. I click on "Proceed to create a dataset"
2. Then "Select all items in study". (30222 items and only 1 patient so far)
(Then the page loads for a very long time ! - I do wait for the page to fully load)
3. After loading, I click on "Save and Define Scope" and at this time, I have no error message and I'm automatically getting back to the "Proceed to create a dataset" page

There is absolutly nothing in the log.

Can you please tell me what to check and what parameters I have to change ?

Thanks a lot for your fast answer.

Romain.




Post edited by RCHENU on
«13

Comments

  • GerbenRienkGerbenRienk Posts: 793 ✭✭✭
    Hi Romain,
    Could this be the same as https://jira.openclinica.com/browse/OC-2056 where tomcat-parameters has been limited to 10.000 ? If so you can try to follow the method described to increase maxParameterCount.
    However I can imagine that the analysis-team would prefer this huge block of data chopped up.
    Kind regards,
    Gerben Rienk
  • RCHENURCHENU Posts: 203
    @Gerben,
    Thanks for your reply.

    I've already changed my server.xml file, so this is not the issue. And I don't have any error message anyway.

    Does it mean that it's not possible to extract all data at once for a big study ?
    Do you think I have to change another parameter ?

    Is it possible in a next version to build an extraction, choosing data event by event and not CRF by CRF for each event ? I'm really stuck right now.

    Thank you,

    Romain.







  • GerbenRienkGerbenRienk Posts: 793 ✭✭✭
    Hi Romain,
    Did you also look at the postgres-logs? On rare occassions something goes wrong there without proper feedback to the application.
    Gerben Rienk
  • RCHENURCHENU Posts: 203
    Thanks Gerben, I'm gonna look tomorrow morning at the postgres-logs. Maybe there is something there.

    When I go Event by Event and CRF by CRf, total items for the whole study is 10548.
    When I do the "Select all items in study" (the method that doesn't work), I have 30222. Do you know why ?

    Thanks,

    Romain.


  • RCHENURCHENU Posts: 203
    @GerbenRienk , can you tell me where and which logs to look at ?
    Thanks.

    Romain.
  • Csaba.HalmagyiCsaba.Halmagyi Posts: 52
    Hi Romain,

    You can find the Postgres logs under /opt/PostgreSQL/8.4/data/pg_log

    Regards,
    Csaba
  • RCHENURCHENU Posts: 203
    Thanks. Logs are empty :-/
  • RCHENURCHENU Posts: 203
    Hi everyone,

    Coming back to this issue: I updated today a lot of my CRFs so I had to upgrade my extraction. Unfortunatly, as it is a big study, I wanted to use the link "add all items in the study" but it doesn't work, I have the same error message that I described at the beginning of this thread (After loading, I click on "Save and Define Scope" and at this time, I have no error message and I'm automatically getting back to the "Proceed to create a dataset" page).

    Please what can I do ? All my daily scripts depend on that extraction and I really need to update it ?
    Any idea, it is quite urgent...

    thanks,

    Romain.


  • kristiakkristiak Posts: 1,318 ✭✭✭
    When you say "updated" do you mean updated the data or updated the CRF design?
  • RCHENURCHENU Posts: 203
    edited July 2017
    For many CRFs, new variables have been added...
Sign In or Register to comment.