Wherever the current working directory is move to ~/cerebmodels.
[1]:
!pwd
/home/harshkhilawala/gsoc/demo_ecosystem/cerebmodels/docs/notebooks
[2]:
cd ..
/home/harshkhilawala/gsoc/demo_ecosystem/cerebmodels/docs
[3]:
cd ..
/home/harshkhilawala/gsoc/demo_ecosystem/cerebmodels
Create MockData Instance to generate mock data¶
[4]:
from cerebstats.test_mock_data import MockData
MD = MockData()
[5]:
# Reference Dataset Link
datasetlink = "https://raw.githubusercontent.com/cerebunit/cerebdata/master/expdata/cells/PurkinjeCell/Llinas_Sugimori_1980_soma_restVm.json"
Generate mock data files with reference dataset¶
[69]:
MD.generate_random_data_files(datasetlink, sample_low=-70,sample_high=-48,num_of_files=10)
Mock Data File 1 generated...
Mock Data File 2 generated...
Mock Data File 3 generated...
Mock Data File 4 generated...
Mock Data File 5 generated...
Mock Data File 6 generated...
Mock Data File 7 generated...
Mock Data File 8 generated...
Mock Data File 9 generated...
Mock Data File 10 generated...
[71]:
MD.count_files()
[71]:
10
[70]:
MD.display_files()
data6.json
data8.json
data9.json
data1.json
data10.json
data3.json
data2.json
data5.json
data4.json
data7.json
[72]:
# Clear out the files
# MD.clear_files()
Load desired Validation Test¶
[73]:
from cerebtests.validation_tests.cells.Purkinje import SomaRestingVmTest as srvt
[74]:
# Set the Model in dictionary format with keys "modelscale" and "modelname"
test_model = {"modelscale": "cells", "modelname": "PC2003Khaliq"}
Instantiate TestMetrics Object¶
[75]:
from cerebstats.test_quality import TestMetrics
[76]:
%%capture
TM = TestMetrics(test_model,srvt, alpha=0.05, epsilon=5)
Calculate the Test Metrics¶
[77]:
TM.calculate_metrics()
Running 1 of 10
Validate Observation ...
dataset is normal
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.09515751699998987 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.14974589499999524 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.1355783710000082 seconds ---
Computing score ...
Done.
u0 = -60.67565845798073 mV, u = -59.30909090909091 mV, n = 55
H0: u = u0
Ha: u =/= u0
Test statistic: z = 0.2301124290815077 dimensionless
Assuming H0 is true, p-value = 0.8180044070217465
Running 2 of 10
Validate Observation ...
dataset is Not normal
dataset is skewed
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.09747960500001795 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.15349269800000798 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.1451948119999997 seconds ---
Computing score ...
Done.
eta0 = -60.67565845798073 mV, eta = -58.0 mV, n = 74
H0: eta = eta0
Ha: eta =/= eta0
Test statistic: z = 2.557448052364024
Assuming H0 is true, p-value = 0.010544330566972694
Running 3 of 10
Validate Observation ...
dataset is Not normal
dataset is Not skewed
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10171535699998913 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.15931845599999406 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.245789142999996 seconds ---
Computing score ...
Done.
eta0 = -60.67565845798073 mV, eta = -59.0 mV, n = 50
H0: eta = eta0
Ha: eta =/= eta0
Test statistic: z = 1.1825266376236836
Assuming H0 is true, p-value = 0.23699680244648247
Running 4 of 10
Validate Observation ...
dataset is normal
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10118826100000433 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.15947674100002018 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.258272912999985 seconds ---
Computing score ...
Done.
u0 = -60.67565845798073 mV, u = -58.48 mV, n = 75
H0: u = u0
Ha: u =/= u0
Test statistic: z = 0.33631258385661145 dimensionless
Assuming H0 is true, p-value = 0.7366351615036643
Running 5 of 10
Validate Observation ...
dataset is normal
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10438432500001227 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.16593286599999146 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.2758500919999847 seconds ---
Computing score ...
Done.
u0 = -60.67565845798073 mV, u = -59.345238095238095 mV, n = 84
H0: u = u0
Ha: u =/= u0
Test statistic: z = 0.20655127348297922 dimensionless
Assuming H0 is true, p-value = 0.8363603176537158
Running 6 of 10
Validate Observation ...
dataset is Not normal
dataset is skewed
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10099187299999812 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.1577753159999986 seconds ---
/envs/py3-cerebmodels/lib/python3.6/site-packages/scipy/stats/stats.py:1604: UserWarning: kurtosistest only valid for n>=20 ... continuing anyway, n=10
"anyway, n=%i" % int(n))
Simulation produce_soma v_restingVm Done.
--- 3.240491643999974 seconds ---
Computing score ...
Done.
eta0 = -60.67565845798073 mV, eta = -60.5 mV, n = 10
H0: eta = eta0
Ha: eta =/= eta0
Test statistic: z = 0.0
Assuming H0 is true, p-value = 1.0
Running 7 of 10
Validate Observation ...
dataset is Not normal
dataset is skewed
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10715595299998881 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.1680948350000051 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.297033875000011 seconds ---
Computing score ...
Done.
eta0 = -60.67565845798073 mV, eta = -52.0 mV, n = 26
H0: eta = eta0
Ha: eta =/= eta0
Test statistic: z = 3.530090432487313
Assuming H0 is true, p-value = 0.00041541764838159116
Running 8 of 10
Validate Observation ...
dataset is Not normal
dataset is Not skewed
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10228512500000875 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.16382761399998458 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.2751330160000123 seconds ---
Computing score ...
Done.
eta0 = -60.67565845798073 mV, eta = -60.0 mV, n = 63
H0: eta = eta0
Ha: eta =/= eta0
Test statistic: z = 0.4655350788461547
Assuming H0 is true, p-value = 0.6415483222069057
Running 9 of 10
Validate Observation ...
dataset is Not normal
dataset is Not skewed
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10897472400000652 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.17115610899998046 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.341652879999998 seconds ---
Computing score ...
Done.
eta0 = -60.67565845798073 mV, eta = -61.5 mV, n = 34
H0: eta = eta0
Ha: eta =/= eta0
Test statistic: z = 1.0172426498607163
Assuming H0 is true, p-value = 0.3090380114196175
Running 10 of 10
Validate Observation ...
dataset is normal
Validated.
Testing ...
Sim produce_soma v_restingVm starting ...
Simulation produce_voltage_response starting ...
--- 0.10844991900000878 seconds ---
File saving ...
File saved.
Simulation produce_voltage_response Done.
--- 0.16898487199998158 seconds ---
Simulation produce_soma v_restingVm Done.
--- 3.349777137999979 seconds ---
Computing score ...
Done.
u0 = -60.67565845798073 mV, u = -59.215053763440864 mV, n = 93
H0: u = u0
Ha: u =/= u0
Test statistic: z = 0.22850703411968248 dimensionless
Assuming H0 is true, p-value = 0.8192520883327371
[78]:
TM.get_specificity()
[78]:
1.0
[79]:
TM.get_sensitivity()
[79]:
0.8888888888888888
[80]:
TM.get_npv()
[80]:
0.5
[81]:
TM.get_ppv()
[81]:
1.0
[82]:
TM.display_outcomes()
+-----------+---------+
| Outcome | Value |
|-----------+---------|
| TP | 8 |
| FP | 0 |
| TN | 1 |
| FN | 1 |
+-----------+---------+
[83]:
TM.display_metrics()
+---------------------------+----------+
| Metric | Value |
|---------------------------+----------|
| Specificity | 1 |
| Sensitivity | 0.888889 |
| Negative Predictive Value | 0.5 |
| Positive Predictive Value | 1 |
+---------------------------+----------+
[ ]:
[ ]: