README.md 8.27 KB
Newer Older
Yunguan Fu's avatar
Yunguan Fu committed
1 2 3 4 5 6 7
# Installation

- `conda create --name py36-tf1 python=3.6 tensorflow-gpu=1.15`
- `conda activate py36-tf1`
- `conda install -c conda-forge opencv`
- `pip install -r requirements.txt`

8 9
# COVID-Net Open Source Initiative

Linda Wang's avatar
Linda Wang committed
10
**Note: The COVID-Net models provided here are intended to be used as reference models that can be built upon and enhanced as new data becomes available. They are currently at a research stage and not yet intended as production-ready models (not meant for direct clinical diagnosis), and we are working continuously to improve them as new data becomes available. Please do not use COVID-Net for self-diagnosis and seek help from your local health authorities.**
11

Linda Wang's avatar
Linda Wang committed
12
**Recording to webinar on [How we built COVID-Net in 7 days with Gensynth](https://darwinai.news/fny)**
lindawangg's avatar
lindawangg committed
13

14
**Update 04/16/2020:** If you have questions, please check the new [FAQ](docs/FAQ.md) page first.\
lindawangg's avatar
FAQ  
lindawangg committed
15 16
**Update 04/15/2020:** We released two new models, COVIDNet-CXR Small and COVIDNet-CXR Large, which were trained on a new COVIDx Dataset with both PA and AP X-Rays from Cohen et al, as well as additional COVID-19 X-Ray images from Figure1.

lindawangg's avatar
lindawangg committed
17
<p align="center">
lindawangg's avatar
lindawangg committed
18
	<img src="assets/covidnet-cxr-small-exp.png" alt="photo not available" width="70%" height="70%">
lindawangg's avatar
lindawangg committed
19 20 21
	<br>
	<em>Example chest radiography images of COVID-19 cases from 2 different patients and their associated critical factors (highlighted in red) as identified by GSInquire.</em>
</p>
lindawangg's avatar
test  
lindawangg committed
22

lindawangg's avatar
lindawangg committed
23
The COVID-19 pandemic continues to have a devastating effect on the health and well-being of the global population.  A critical step in the fight against COVID-19 is effective screening of infected patients, with one of the key screening approaches being radiological imaging using chest radiography.  It was found in early studies that patients present abnormalities in chest radiography images that are characteristic of those infected with COVID-19.  Motivated by this, a number of artificial intelligence (AI) systems based on deep learning have been proposed and results have been shown to be quite promising in terms of accuracy in detecting patients infected with COVID-19 using chest radiography images.  However, to the best of the authors' knowledge, these developed AI systems have been closed source and unavailable to the research community for deeper understanding and extension, and unavailable for public access and use.  Therefore, in this study we introduce COVID-Net, a deep convolutional neural network design tailored for the detection of COVID-19 cases from chest radiography images that is open source and available to the general public.  We also describe the chest radiography dataset leveraged to train COVID-Net, which we will refer to as COVIDx and is comprised of 13,800 chest radiography images across 13,725 patient patient cases from three open access data repositories.  Furthermore, we investigate how COVID-Net makes predictions using an explainability method in an attempt to gain deeper insights into critical factors associated with COVID cases, which can aid clinicians in improved screening.  **By no means a production-ready solution**, the hope is that the open access COVID-Net, along with the description on constructing the open source COVIDx dataset, will be leveraged and build upon by both researchers and citizen data scientists alike to accelerate the development of highly accurate yet practical deep learning solutions for detecting COVID-19 cases and accelerate treatment of those who need it the most.
lindawangg's avatar
lindawangg committed
24

lindawangg's avatar
lindawangg committed
25
For a detailed description of the methodology behind COVID-Net and a full description of the COVIDx dataset, please click [here](https://arxiv.org/abs/2003.09871v3).
26

27
Currently, the COVID-Net team is working on **COVID-RiskNet**, a deep neural network tailored for COVID-19 risk stratification.  Currently this is available as a work-in-progress via included `train_risknet.py` script, help to contribute data and we can improve this tool.
lindawangg's avatar
lindawangg committed
28

29
If you would like to **contribute COVID-19 x-ray images**, please submit to https://figure1.typeform.com/to/lLrHwv. Lets all work together to stop the spread of COVID-19!
lindawangg's avatar
lindawangg committed
30

lindawangg's avatar
lindawangg committed
31
If you are a researcher or healthcare worker and you would like access to the **GSInquire tool to use to interpret COVID-Net results** on your data or existing data, please reach out to a28wong@uwaterloo.ca or alex@darwinai.ca
lindawangg's avatar
lindawangg committed
32

lindawangg's avatar
lindawangg committed
33
Our desire is to encourage broad adoption and contribution to this project. Accordingly this project has been licensed under the GNU Affero General Public License 3.0. Please see [license file](LICENSE.md) for terms. If you would like to discuss alternative licensing models, please reach out to us at linda.wang513@gmail.com and a28wong@uwaterloo.ca or alex@darwinai.ca
lindawangg's avatar
lindawangg committed
34

lindawangg's avatar
FAQ  
lindawangg committed
35
If there are any technical questions after the README, FAQ, and past/current issues have been read, please post an issue or contact:
Desmond Lin's avatar
Desmond Lin committed
36 37 38
* desmond.zq.lin@gmail.com
* paul@darwinai.ca
* jamesrenhoulee@gmail.com
lindawangg's avatar
lindawangg committed
39
* linda.wang513@gmail.com
lindawangg's avatar
lindawangg committed
40
* ashkan.ebadi@nrc-cnrc.gc.ca
lindawangg's avatar
lindawangg committed
41

lindawangg's avatar
lindawangg committed
42 43 44 45 46
If you find our work useful, can cite our paper using:

```
@misc{wang2020covidnet,
    title={COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest Radiography Images},
lindawangg's avatar
lindawangg committed
47
    author={Linda Wang, Zhong Qiu Lin and Alexander Wong},
lindawangg's avatar
lindawangg committed
48 49 50 51 52 53 54
    year={2020},
    eprint={2003.09871},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
```

lindawangg's avatar
lindawangg committed
55 56 57 58 59 60 61
## Core COVID-Net Team
* DarwinAI Corp., Canada and Vision and Image Processing Research Group, University of Waterloo, Canada
	* Linda Wang
	* Alexander Wong
	* Zhong Qiu Lin
	* Paul McInnis
	* Audrey Chung
Linda Wang's avatar
Linda Wang committed
62
	* Hayden Gunraj, COVIDNet for CT: Coming soon.
Linda Wang's avatar
Linda Wang committed
63
* Vision and Image Processing Research Group, University of Waterloo, Canada
lindawangg's avatar
lindawangg committed
64
	* James Lee
lindawangg's avatar
lindawangg committed
65 66 67 68 69 70 71 72 73 74 75 76
* Matt Ross and Blake VanBerlo (City of London), COVID-19 Chest X-Ray Model: https://github.com/aildnont/covid-cxr
* Ashkan Ebadi (National Research Council Canada)
* Kim-Ann Git (Selayang Hospital)
* Abdul Al-Haimi

## Table of Contents
1. [Requirements](#requirements) to install on your system
2. How to [generate COVIDx dataset](docs/COVIDx.md)
3. Steps for [training, evaluation and inference](docs/train_eval_inference.md)
4. [Results](#results)
5. [Links to pretrained models](docs/models.md)

lindawangg's avatar
lindawangg committed
77
## Requirements
78 79 80

The main requirements are listed below:

lindawangg's avatar
lindawangg committed
81 82
* Tested with Tensorflow 1.13 and 1.15
* OpenCV 4.2.0
lindawangg's avatar
lindawangg committed
83
* Python 3.6
84 85 86 87 88 89
* Numpy
* Scikit-Learn
* Matplotlib

Additional requirements to generate dataset:

90
* PyDicom
91 92
* Pandas
* Jupyter
lindawangg's avatar
lindawangg committed
93

94
## Results
lindawangg's avatar
lindawangg committed
95
These are the final results for COVIDNet-CXR Small and COVIDNet-CXR Large.
lindawangg's avatar
lindawangg committed
96

lindawangg's avatar
lindawangg committed
97
### COVIDNet-CXR Small
Linda Wang's avatar
Linda Wang committed
98
<p>
lindawangg's avatar
lindawangg committed
99
	<img src="assets/cm-covidnetcxr-small.png" alt="photo not available" width="50%" height="50%">
100
	<br>
lindawangg's avatar
lindawangg committed
101
	<em>Confusion matrix for COVIDNet-CXR Small on the COVIDx test dataset.</em>
102
</p>
lindawangg's avatar
lindawangg committed
103

Linda Wang's avatar
Linda Wang committed
104
<div class="tg-wrap"><table class="tg">
105 106 107 108 109 110 111 112 113
  <tr>
    <th class="tg-7btt" colspan="3">Sensitivity (%)</th>
  </tr>
  <tr>
    <td class="tg-7btt">Normal</td>
    <td class="tg-7btt">Pneumonia</td>
    <td class="tg-7btt">COVID-19</td>
  </tr>
  <tr>
lindawangg's avatar
lindawangg committed
114 115 116
    <td class="tg-c3ow">97.0</td>
    <td class="tg-c3ow">90.0</td>
    <td class="tg-c3ow">87.1</td>
117 118
  </tr>
</table></div>
lindawangg's avatar
lindawangg committed
119

Linda Wang's avatar
Linda Wang committed
120
<div class="tg-wrap"><table class="tg">
121 122 123 124 125 126 127 128 129
  <tr>
    <th class="tg-7btt" colspan="3">Positive Predictive Value (%)</th>
  </tr>
  <tr>
    <td class="tg-7btt">Normal</td>
    <td class="tg-7btt">Pneumonia</td>
    <td class="tg-7btt">COVID-19</td>
  </tr>
  <tr>
lindawangg's avatar
lindawangg committed
130 131 132
    <td class="tg-c3ow">89.8</td>
    <td class="tg-c3ow">94.7</td>
    <td class="tg-c3ow">96.4</td>
133 134 135
  </tr>
</table></div>

lindawangg's avatar
lindawangg committed
136 137

### COVIDNet-CXR Large
Linda Wang's avatar
Linda Wang committed
138
<p>
lindawangg's avatar
lindawangg committed
139
	<img src="assets/cm-covidnetcxr-large.png" alt="photo not available" width="50%" height="50%">
lindawangg's avatar
lindawangg committed
140
	<br>
lindawangg's avatar
lindawangg committed
141
	<em>Confusion matrix for COVIDNet-CXR Large on the COVIDx test dataset.</em>
lindawangg's avatar
lindawangg committed
142 143
</p>

Linda Wang's avatar
Linda Wang committed
144
<div class="tg-wrap"><table class="tg">
lindawangg's avatar
lindawangg committed
145
  <tr>
146
    <th class="tg-7btt" colspan="3">Sensitivity (%)</th>
lindawangg's avatar
lindawangg committed
147 148 149
  </tr>
  <tr>
    <td class="tg-7btt">Normal</td>
150 151
    <td class="tg-7btt">Pneumonia</td>
    <td class="tg-7btt">COVID-19</td>
lindawangg's avatar
lindawangg committed
152 153
  </tr>
  <tr>
lindawangg's avatar
lindawangg committed
154 155 156
    <td class="tg-c3ow">99.0</td>
    <td class="tg-c3ow">89.0</td>
    <td class="tg-c3ow">96.8</td>
lindawangg's avatar
lindawangg committed
157 158 159
  </tr>
</table></div>

Linda Wang's avatar
Linda Wang committed
160
<div class="tg-wrap"><table class="tg">
lindawangg's avatar
lindawangg committed
161
  <tr>
162
    <th class="tg-7btt" colspan="3">Positive Predictive Value (%)</th>
lindawangg's avatar
lindawangg committed
163 164 165
  </tr>
  <tr>
    <td class="tg-7btt">Normal</td>
166 167
    <td class="tg-7btt">Pneumonia</td>
    <td class="tg-7btt">COVID-19</td>
lindawangg's avatar
lindawangg committed
168 169
  </tr>
  <tr>
lindawangg's avatar
lindawangg committed
170 171 172
    <td class="tg-c3ow">91.7</td>
    <td class="tg-c3ow">98.9</td>
    <td class="tg-c3ow">90.9</td>
lindawangg's avatar
lindawangg committed
173 174
  </tr>
</table></div>