=================
JSON requirements
=================

JSON files are the machine-readable results that will be used throughout the DQR to generate result pages, track progress, and connect with other tools. All JSON files should be named `data.json` in order to be properly identified by the results page task. Inside the file, there is a set of required and optional fields that will be parsed by other DQR processes. If other fields are included in JSON result files, this should not cause errors, but there is no expectation that these fields will be used by other DQR processes.  

Required fields
---------------

There is a set of required JSON fields for all tasks: 

.. list-table:: Required fields
   :header-rows: 1

   * - Field
     - Data Type
     - Definition
     - Allowed Values
     - Default Value
   * - `date` 
     - `str`
     - Datetime value of when JSON was generated.
     - ISO-formatted date-times
     - '' (empty string)
   * - `hostname`
     - `str`
     - Hostname of machine used to generate result.
     - Any string
     - '' (empty string)
   * - `process_name`
     - `str`
     - Name of DQR process for this result.
     - Any string
     - '' (empty string)
   * - `process_version`
     - `str`
     - Version of DQR process used to generate this result, corresponding to the version-controlled source of this DQR process's code.
     - Any string
     - '' (empty string)
   * - `state`
     - `str`
     - State of the DQR process when this JSON was generated.
     - One of the following: bad_state / default / error / fail / human_input_needed / irrelevant / missing / pass / done / not_done. 
     - '' (empty string)
   * - `summary`
     - `str`
     - Executive summary of the DQR result. Should include an actionable result if DQR process is complete.
     - Any string
     - '' (empty string)
   * - `username`
     - `str`
     - Username associated with the account that generated this JSON.
     - Any string
     - '' (empty string)

All DQR tasks must report their results in the above format to receive review approval. 

Optional fields
---------------

There is also a set of optional fields that processes can use when reporting their results:  

.. list-table:: Optional fields
   :header-rows: 1

   * - Field
     - Data Type
     - Definition
     - Allowed Values
     - Default Value
   * - `p_value`
     - `float`
     - P value reported by the DQR process. Required for automatic parsing of process Pass/Fail.
     - Any value in [0,1] or -1
     - -1
   * - `extra` 
     - `list`
     - Suggested location of additional fields, stored as a list of dictionaries. 
     - List of dictionaries, no required format
     - [] (empty list)
   * - `directory`
     - `str`
     - Directory containing results of this DQR process.
     - path-like object
     - '' (empty string)
   * - `librarian`
     - `str`
     - Code maintainer of this DQR process.
     - Any string
     - '' (empty string)
   * - `figures`
     - `list`
     - List of figures associated with this DQR process. Required for parsing of O3-style JSON result files. 
     - "List of dictionaries. Each figure directory has the following format: `{"src": "[<relative path>/]<filename>", "alt": "<Description>", "width": <integer>}`	
     - [] (empty list)
   * - `links`
     - `list`
     - List of links associated with this DQR process. Required for parsing of O3-style JSON result files.
     - "List of dictionaries. Each link directory has the following format: `{"href": "[<relative path>/]<filename>", "innerHTML": "Description"}`
     - [] (empty string)
   * - `tables`
     - `list`
     - List of tables associated with this DQR process. Required for parsing of O3-style JSON result files.
     - "List of dictionaries. A table dictionary includes two keys: "columns" and "data". "columns" is the list of column names. "data" is a list of rows, each row is a list giving the contents of the row, column by column. If extra fields are included (such as a "states" field) this will not cause errors.  	
     - [] (empty list)
   * - `stat_value`
     - `float`
     - Value of statistic reported by this DQR process. 
     - Any float
     - -1

While optional, some of these fields are required for certain automated processes, such as generated Pass/Fail labels or connecting with O3-back-compatibility tools. 

JSON example
------------

An example of a complete JSON file containing all fields is below: 

.. code-block:: json

    {
        "date": "2023-03-20 12:30:00 UTC", 
        "hostname": "olnode13.virgo.infn.it",
        "process_name": "dqr-bristol",
        "process_version": "0.1",
        "state": "human_input_needed", 
        "summary": "BRISTOL (Band-Rms Stationarity Test toOL) results channel by channel. Human input is needed.", 
        "p-value": "-1",
        "extra": [],
        "directory": "/path/to/result/directory/",
        "librarian": "Albert Einstein (albert.einstein@ligo.org)", 
        "figures": [{"src": "bristol_v1_Hrec_hoft_16384Hz_INJ1_O3Replay.png", "width": 650}],
        "links": [{"href": "https://dcc.ligo.org", "innerHTML": "a link to the DCC which should contain some documentation for this process"}], 
        "tables": [{"columns": ["active time", "duration", "active fraction", "activity threshold"],
                    "data": [ [0.0, 10.0, 0.0, 0.0], [0.1, 10.0, 0.01, 0.5] ]}
                  ], 
        "stat_value": "-1"
    }