We hope you'll join us for our 4/23 webinar on using data tables to apply reference ranges and AE codes in OC4. For more information and to register, visit https://register.gotowebinar.com/register/2882170018956684555

Best practice for entering time series data

Hi,

I am collecting time series data over a longer period of time (several weeks) for my study and would like to submit the data as it arrives from my sensors in real-time to my OpenClinica instance using the API. Is there any recommended best practice on how to achieve this?

Any advice or recommendation is appreciated!

Comments

  • GerbenRienkGerbenRienk Posts: 838 ✭✭✭
    Hi,
    Do you mean you can access an api to get the sensor-data? If so and if the format is json or xml it would be similar to reading data of LimeSurvey. On my github you can find a python-implementation for that:
    https://github.com/GerbenRienk/oli_est where "oli" stands for OpenClinica-LimeSurvey-Interface. It loops with an interval you can set and sends a request to the api, transforms the data into oc-readable format and at the end of the day sends a report/log.
    Is this more or less what you were thinking about?
    Kind regards,
    Gerben Rienk
  • medonjamedonja Posts: 5
    Hi Gerben,

    thanks for taking the time to reply! That sounds indeed like an interesting approach! But I am not sure if it can be compatible with my need, as I would like to push the data to OpenClinica, instead of pulling the data from my subjects' sensors... :/ The format is though indeed JSON or XML. But yes, your solution is indeed close to what I was thinking of. I will have a more detailed look into oli_est in the next couple of days and get back to you.

    Moreover, I am not sure if it is possible to acquire sensor data periodically from a sensor without adding entries to the audit log? I am worried that if I generate data every couple of seconds from a sensor that the audit log could get very fast bloated with entries. On the other hand, storage space is nowadays very cheap.

    Best and thanks,
    Marko
  • toskriptoskrip Posts: 279 ✭✭✭
    Hi Marko,

    I am not sure if I would reccomend you to store time series data from sensors in eCRF structure in OC. It very much depends on the amount of data (number of timestamps), if you want to keep all timestamps or only the last one etc. But I can tell you that thare are limits on e.g. how many repeating items you can have in eCRF grid strucutre. You will definitelly need to do some tests to see but in my opinnion OC is not suited for this type of data collection. You would be probably better of storing those datasets in different system (JSON or XML document store) and in OC only reference to those datasets.

    best

    Tomas
  • medonjamedonja Posts: 5
    Hi Tomas,

    yes, that is exactly the concern I am having. Well, at least now I have some first-hand OpenClinica expert feedback and a starting point to continue to look for a suitable solution for my issue.

    Best,
    Marko
Sign In or Register to comment.