Context of processing in SIOS

Summary of SIOS requirements for processing

Detailed requirements

1.Data processing desiderata: input

 i.   What data are to be processed? What are their:

• Typologies
• Volume
• Velocity
• Variety

 [can’t answer

ii.   How is the data made available to the analytics phase? By file, by web (stream/protocol), etc.

 [can’t answer]

iii.  Please provide concrete examples of data.

 [can’t answer]

2. Data processing desiderata: analytics

i.   Computing needs quantification:

• How many processes do you need to execute?
• How much time does each process take/should take?

 [can’t answer]

 ii.  Process implementation:

• What do you use in terms of:

○      Programming languages?

○      Platform?

○      Specific software requirements?

[can’t answer]

• What standards need to be supported (e.g. WPS) for each of the above?

[Answers here]

• Is there a possibility to inject proprietary/user defined algorithms/processes for each of the above?

[Answers here]

• Do you use a sandbox to test and tune the algorithm/process for each of the above?

[Answers here]

 iii.  Do you use batch or interactive processing?

[Answers here]

iv.  Do you use a monitoring console?

[Answers here]

v.   Do you use or black box or workflow processing?

• Do you reuse sub-processes across processes?

[Answers here]

vi.   Please provide concrete examples of processes to be supported/currently in use;

[Answers here]

3.     Data processing desiderata: output

i.   What data are produced? Please provide:

• Typologies
• Volume
• Velocity
• Variety

[Answers here]

ii.   How are analytics outcomes made available?

[Answers here]

4.     Statistical questions

i.    Is the data collected with a distinct question/hypothesis in mind? Or is simply something being measured?

[Answers here]

ii.   Will questions/hypotheses be generated or refined (broadened or narrowed in scope) after the data has been collected? (N.B. Such activity would not be good statistical practice)

[Answers here]

5. Statistical data

i.   Does the question involve analysing the responses of a single set of data (univariate) to other predictor variables or are there multiple response data (bi or multivariate data)?

[Answers here]

ii.   Is the data continuous or discrete?

[Answers here]

iii.  Is the data bounded in some form (i.e. what is the possible range of the data)?

iv.  Typically how many datums approximately are there?

[Answers here]

6.  Statistical data analysis

i.   Is it desired to work within a statistics or data mining paradigm? (N.B. the two can and indeed should overlap!)

[Answers here]

ii.   Is it desired that there is some sort of outlier/anomaly assessment?

[Answers here]

iii.   Are you interested in a statistical approach which rejects null hypotheses (frequentist) or generates probable belief in a hypothesis (Bayesian approach) or do you have no real preference?

 [Answers here]

Formalities (who & when)