...
Step | Description of action | Dependency on 3rd party services (EOSC-hub or other) |
|---|---|---|
UC1 | User needs to discover the location of all required input data | ESGF Metadata Service/B2FIND |
UC2 | Input data must have a PID associated with it. | Community solutions assigning PIDs, possibly via B2HANDLE? |
UC3 | ENES Data Analytics Service must be able to transfer data from its current location to the processing site based on PID (Low priority - I am not sure if we will do this; it is not entirely in the original plan, though I agree it makes sense. It depends on how data input integration ultimately looks like and what can be done with limited effort.) | gridFTP/other? |
| UC4 | Output data must be moved to a site where users can share it for others so they can access it via a link provided by the ECAS system. | B2DROP |
| UC5 | Users will need to register to use the ECAS service | Appropriate EOSC-AAI Solution |
| UC6 | Data must be movable between the output storage in UC4 to a data publication service, where it must be given appropriate metadata and a PID | B2SHARE |
| UC7 | Output data shall have appropriate and sufficient metadata and provenance information associated to enable other users to have trust in the data. | ECAS?, B2HANDLE profiles (possibly their usage by B2DROP) |
| UC8 | A link between the output data and the sources must be maintained, in addition to provenance information related to the processing steps. | ECAS?, B2HANDLE profiles (possibly their usage by B2DROP) |
| UC9 | Input data must be accessible to the computation regardless of location. | B2HANDLE ?usage by communities and the DataHub |
| UC10 | Published output data must be assigned a PID | B2SHARE |
| UC11 | The provenance information must be accessible for published output data | B2SHARE usage of B2HANDLE profiles |
Requirements
Technical Requirements
...
Requirement ID | EOSC-hub service | GAP (Yes/No) + description | Requirement description | Source Use Case |
|---|---|---|---|---|
Example | EOSC-hub AAI | Yes: EOSC-hub AAI doesn’t support the Marine IdP | EOSC-hub AAI should accept Marine IDs | UC1 |
RQ1 | EOSC-hub AAI | ESGF AAI not integrated to any AAI services | Integration of ESGF AAI to one of EOSC AAI services | UC5 |
RQ2 | B2DROP | Can be a central service; no need for local installation. User has no interface to B2DROP filesystem; currently user log in to jupyter with username and password. Files automatically moved to B2DROP without user intervention. GAP: Need to integrate AAI to B2DROP . For training purposes, cosider consider using a proxy user for training purposes. | Need to be able to write directly to B2DROP (via mount point inaccessible to users), or have the workflow copy data in using NextCloud OpenCloudMesh API. Will require separate instances for training and production | UC4 |
| RQ3 | B2DROP | GAP - UNSURE - If data is moved using OpenCloudMesh, the security needs to be considered. NextCloud website recommends using SSL since user information is passed in plain text. Need to check how B2DROP is configured. | B2DROP must run with SSL enabled | UC4 |
| RQ4 | B2SHARE | GAP - NO (if RQ2 is satified), YES (otherwise) Enable users to push files to B2SHARE. If RQ2 works there is no gap to deal with as the bridge exists. Unless RQ2 works, then beed to integrate AAI to B2SHARE | B2DROP/B2SHARE Bridge required | UC6 |
| RQ5 | Datahub | GAP - UNCLEAR Data publishing and data ingest. Allows contacting multiple communities. | ||
| RQ6 | B2HANDLE | GAP - NoUNCLEAR Both input data and published derived data must be assigned a PID. For third-party users to access provenance information, B2SHARE and possibly also B2DROP need to support recording of minimal provenance information, possibly organized via B2HANDLE profiles. | UC7, UC8, UC11 |
Capacity Requirements
EOSC-hub services | Amount of requested resources | Time period |
|---|---|---|
| B2DROP | Testing & Training (200GB, <1GB/file) | M5 onwards |
| B2DROP | Production (see table, >500MB) | M7 onwards |
| B2SHARE | Testing & Training ((200GB, <1GB/file) | M5 onwards |
| B2SHARE | Production (see table, >500MB) | M7 ideally, M12 latest |
| B2HANDLE | Production, 2-4 prefixes required (CMCC, DKRZ, EGI, spare) | M15 - |
| DataHub | Unknow | M15 |
| IM/Orchestrator | Unknown | M18- |