Commit dffd037a authored by mathpluscode's avatar mathpluscode

add readme and correct bugs

parent a368d7b2
Pipeline #3242 failed with stages
in 2 minutes and 22 seconds
====================
yunguanfu-mil3id2019
===============================
====================
.. image:: https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/yunguanfu-mil3id2019/raw/master/project-icon.png
:height: 128px
......@@ -27,58 +28,93 @@ yunguanfu-mil3id2019 is part of the `SNAPPY`_ software project, developed at the
yunguanfu-mil3id2019 supports Python 3.6.
yunguanfu-mil3id2019 is currently a demo project, which will add/multiply two numbers. Example usage:
yunguanfu-mil3id2019 is currently a demo project. It implements Yunguan Fu's paper `More unlabelled data or label more data? A study on semi-supervised laparoscopic image segmentation`_.
Example Usage
-------------
Preprocess
^^^^^^^^^^
::
python yfmil3id2019_train.py -x 5 -y 8
python yfmil3id2019_train.py -x 3 -y 6 --multiply
python preprocess_mean_std.py -p config.yaml
Please explore the project structure, and implement your own functionality.
This code calculates the mean and std on channel level for the image data of each patient. The output is stored in ``data/img/mean_std``. These files are required for training.
Developing
----------
::
Cloning
^^^^^^^
python preprocess_unlabel.py -f 4
This code extracts the unlabelled images from videos. The output is stored in ``data/img/unlabeled``. These files are required for semi-supervised training.
::
python preprocess_cut.py -p config.yaml
This code calculates the images without black border. The output is stored in ``data/img/cut``. These files are required for evaluation.
Train
^^^^^
You can clone the repository using the following command:
::
python train.py -p config.yaml -g 0
This code trains models with parameters defined in config.yaml file and uses GPU number 0.
Evaluate
^^^^^^^^
::
git clone https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/yunguanfu-mil3id2019
python eval.py -p log/20190902225640-config -g 0 --eval
This code evaluates the models in the log folder. It calculates the prediction of each model of cross validation on the corresponding test set. The predictions of ``LR05`` will be saved in ``log/20190902225640-config/LR05/preds/final``.
Running tests
^^^^^^^^^^^^^
Pytest is used for running unit tests:
::
pip install pytest
python -m pytest
python eval.py -p log/20190902225640-config -g 0 --eval --all
This code calculates the prediction of each model of cross validation on all data, including training set and test set. The predictions of ``LR05`` will be saved in ``log/20190902225640-config/LR05/preds/final_all``.
Linting
Analyse
^^^^^^^
This code conforms to the PEP8 standard. Pylint can be used to analyse the code:
::
python analyse.py -p log/ --std
This code
* summarises the statistics of metrics
* generates the performance vs foreground proportion curve only for test set.
::
pip install pylint
pylint --rcfile=tests/pylintrc yfmil3id2019
python analyse.py -p log/ --std --all
This code
Installing
----------
* summarises the statistics of metrics
* generates the performance vs foreground proportion curves, one for training and another one for test.
But it requires to evaluate with ``--all``
You can pip install directly from the repository as follows:
Test
^^^^
::
pip install git+https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/SNAPPY/yunguanfu-mil3id2019
python test.py -p log/20190902225640-config/LR05 -d demo -g 0
This code tests the model in the folder ``log/20190902225640-config/LR05`` on images inside ``demo/``.
Developing
----------
Contributing
^^^^^^^^^^^^
......@@ -115,4 +151,4 @@ Supported by `Wellcome`_ and `EPSRC`_.
.. _`EPSRC`: https://www.epsrc.ac.uk/
.. _`contributing guidelines`: https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/yunguanfu-mil3id2019/blob/master/CONTRIBUTING.rst
.. _`license file`: https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/yunguanfu-mil3id2019/blob/master/LICENSE
.. _`More unlabelled data or label more data? A study on semi-supervised laparoscopic image segmentation`: https://arxiv.org/abs/1908.08035
......@@ -63,7 +63,7 @@ def analyse_foreground(log_path, best, std):
metrics_total = dict()
for p in folder_paths:
name = ''.join(p.split('-')[1:])
m_total, _, _ = read_folder(p, best=best, total=False)
m_total, _, _ = read_folder(p, best=best, total=False, all=False)
names.append(name)
metrics_total[name] = m_total
......
......@@ -111,16 +111,11 @@ def calculate_metrics(fnames, dir_cut, return_hd, all):
def eval_app(config, path, best, all, eval, return_hd):
# create log folder
app_name = 'eval-' + path.split('/')[-1]
cwd = os.getcwd() + '/'
dir_log = init_log_dir(cwd + config['dir']['log'], app_name)
set_tf_logger(dir_log, app_name)
# check if the copy of cut data has been made
dir_cut = config['dir']['data'] + 'img/cut/'
# get folders for labeled and unlabeled data
cwd = os.getcwd() + '/'
fs_lbl = get_labeled_folders(cwd, config)
fs_unlbl = get_unlabeled_folders(fs_lbl, config)
verify_dirs(fs_unlbl)
......
......@@ -50,12 +50,6 @@ def test_app(config, model_path, data_path, best):
:param best:
:return:
"""
# create log folder
app_name = 'test'
cwd = os.getcwd() + '/'
dir_log = init_log_dir(cwd + config['dir']['log'], app_name)
set_tf_logger(dir_log, app_name)
# init configs
session_config = ConfigProto(device_count={'GPU': 0})
run_config = tf.estimator.RunConfig(session_config=session_config, **config['tf']['run'])
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment