Commit 60cee1cb authored by Matt Clarkson's avatar Matt Clarkson

Issue #1: Implement a script to export masks from Labelbox.

parent 18d23ea9
......@@ -80,42 +80,6 @@ test Windows:
tags:
- shared-win
# Template for deploy stages that depend on 'build docs'
# if the build stage is skipped, skip these also
.deploy-docs-skip-template: &deploy-docs-skip-template
stage: deploy
when: manual
except:
variables:
- $CI_COMMIT_MESSAGE =~ /\[skip[ _-]build?\]/i
dependencies:
- build docs
deploy docs to staging:
<<: *deploy-docs-skip-template
script:
# Note: the group/username directory must already exist on the server before calling this command
- rsync -avz -e'ssh -v' --numeric-ids --delete doc/build/html/* staging_docs_rsync:WEISS/SoftwareRepositories/LabelboxUtils 2>&1
tags:
- docs-staging
environment:
name: staging
url: http://weisslab-lin.cs.ucl.ac.uk/staging/WEISS/SoftwareRepositories/LabelboxUtils
only:
- master
deploy docs to production:
<<: *deploy-docs-skip-template
script:
# Note: the group/username directory must already exist on the server before calling this command
- rsync -avz -e'ssh -v' --numeric-ids --delete doc/build/html/* production_docs_rsync:WEISS/SoftwareRepositories/LabelboxUtils 2>&1
tags:
- docs-production
environment:
name: production
only:
- public
deploy pip to PyPI:
stage: deploy
when: manual
......
Labelbox Utils
===============================
==============
.. image:: https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils/raw/master/project-icon.png
:height: 128px
......@@ -20,21 +20,15 @@ Labelbox Utils
:alt: Documentation Status
Author: Matt Clarkson
Labelbox Utils is part of the `SNAPPY`_ software project, developed at the `Wellcome EPSRC Centre for Interventional and Surgical Sciences`_, part of `University College London (UCL)`_.
Labelbox Utils supports Python 2.7 and Python 3.6.
LabelboxUtils contains python scripts for working with Labelbox. Features so
far include:
Labelbox Utils is currently a demo project, which will add/multiply two numbers. Example usage:
* ```labelbox_export_json```: Takes .json file and extracts masks.
::
For command line apps, run with ```-h``` to see more usage information.
python labelboxutils.py 5 8
python labelboxutils.py 3 6 --multiply
Please explore the project structure, and implement your own functionality.
Developing
----------
......@@ -51,22 +45,16 @@ You can clone the repository using the following command:
Running tests
^^^^^^^^^^^^^
Pytest is used for running unit tests:
::
pip install pytest
python -m pytest
Pytest is used for running unit tests, but you should run using tox,
as per the `PythonTemplate`_ instructions.
Linting
^^^^^^^
This code conforms to the PEP8 standard. Pylint can be used to analyse the code:
::
pip install pylint
pylint --rcfile=tests/pylintrc labelboxutils
This code conforms to the PEP8 standard. Pylint is used to analyse the code.
Again, follow the `PythonTemplate`_ instructions and run via tox.
Installing
......@@ -79,15 +67,14 @@ You can pip install directly from the repository as follows:
pip install git+https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils
Contributing
^^^^^^^^^^^^
------------
Please see the `contributing guidelines`_.
Useful links
^^^^^^^^^^^^
------------
* `Source code repository`_
* `Documentation`_
......@@ -109,7 +96,6 @@ Supported by `Wellcome`_ and `EPSRC`_.
.. _`Wellcome EPSRC Centre for Interventional and Surgical Sciences`: http://www.ucl.ac.uk/weiss
.. _`source code repository`: https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils
.. _`Documentation`: https://LabelboxUtils.readthedocs.io
.. _`SNAPPY`: https://weisslab.cs.ucl.ac.uk/WEISS/PlatformManagement/SNAPPY/wikis/home
.. _`University College London (UCL)`: http://www.ucl.ac.uk/
.. _`Wellcome`: https://wellcome.ac.uk/
.. _`EPSRC`: https://www.epsrc.ac.uk/
......
......@@ -5,9 +5,7 @@ Contents
.. toctree::
:maxdepth: 1
requirements
Module Contents <modules>
notebooks/first_notebook
* :ref:`modindex`
* :ref:`genindex`
......
.. highlight:: shell
.. _requirements:
===============================================
Requirements for Labelbox Utils
===============================================
This is the software requirements file for Labelbox Utils, part of the
SNAPPY project. The requirements listed below should define
what Labelbox Utils does. Each requirement can be matched to a unit test that
checks whether the requirement is met.
Requirements
~~~~~~~~~~~~
+------------+--------------------------------------------------------+-------------------------------------+
| ID | Description | Test |
+============+========================================================+=====================================+
| 0000 | Module has a help page | pylint, see |
| | | tests/pylint.rc and tox.ini |
+------------+--------------------------------------------------------+-------------------------------------+
| 0001 | Functions are documented | pylint, see |
| | | tests/pylint.rc and tox.ini |
+------------+--------------------------------------------------------+-------------------------------------+
| 0002 | Package has a version number | No test yet, handled by git. |
+------------+--------------------------------------------------------+-------------------------------------+
......@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-
import sys
from labelboxutils.ui.labelboxutils_command_line import main
from labelboxutils.ui.labelbox_export_json_command_line import main
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""LabelboxUtils"""
import sys
if __name__ == "__main__" and not __package__:
# To allow the package's main function to be executed without the -m switch,
# i.e. "python labelboxutils", we have to explicitly set the
# module name and append the parent directory to the sys.path (see PEP 366)
from os import path
__package__ = "labelboxutils" # pylint: disable=redefined-builtin
sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
sys.path.append(path.dirname(path.dirname(__file__)))
# pylint: disable=wrong-import-position
from labelboxutils.ui.labelboxutils_command_line import main
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
# coding=utf-8
""" Module for adding numbers. """
def add_two_numbers(input_x, input_y):
""" Add two numbers """
return input_x + input_y
# coding=utf-8
""" Module for multiplying numbers. """
def multiply_two_numbers(input_x, input_y):
""" Multiply two numbers """
return input_x * input_y
# coding=utf-8
""" Processing for labelbox_export_json appliction. """
import os
import json
import urllib.request
import six
import cv2
def run_labelbox_json_export(json_file, output_dir, output_format):
"""
Parses the Labelbox json file, and exports data.
:param json_file: file name of json file
:param output_dir: where to write to
:param output_format: output image format, [grey|rgb|rgba (default)]
"""
if not os.path.exists(output_dir):
os.makedirs(output_dir)
with open(json_file) as file:
data = file.read()
decoded_data = json.loads(data)
for sample in decoded_data:
dataset_name = sample['Dataset Name']
external_id = sample['External ID']
basename = os.path.basename(external_id)
filename_no_extension = os.path.splitext(basename)[0]
objects = sample['Label']['objects']
dir_name = os.path.join(output_dir, dataset_name)
if not os.path.exists(dir_name):
os.makedirs(dir_name)
for obj in objects:
instance_uri = obj['instanceURI']
value = obj['value']
file_name = os.path.join(dir_name,
filename_no_extension
+ '_' + value + '.bmp')
six.print_("writing to:" + file_name)
urllib.request.urlretrieve(instance_uri, file_name)
if output_format == 'grey':
image = cv2.imread(file_name)
grey = cv2.cvtColor(image, cv2.COLOR_BGRA2GRAY)
file_name = os.path.join(dir_name,
filename_no_extension
+ '_' + value + '.png')
cv2.imwrite(file_name, grey)
six.print_("writing to:" + file_name)
elif output_format == 'rgb':
image = cv2.imread(file_name)
rgb = cv2.cvtColor(image, cv2.COLOR_BGRA2BGR)
file_name = os.path.join(dir_name,
filename_no_extension
+ '_' + value + '.png')
cv2.imwrite(file_name, rgb)
six.print_("writing to:" + file_name)
# coding=utf-8
""" Command line processing for labelbox_export_json. """
import argparse
from labelboxutils import __version__
from labelboxutils.ui.labelbox_export_json_app import run_labelbox_json_export
def main(args=None):
""" Entry point for labelbox_export_json application. """
parser = argparse.ArgumentParser(description='LabelBox JSON Export')
parser.add_argument("-j", "--json",
required=True,
type=str,
help="JSON file exported from Labelbox.")
parser.add_argument("-o", "--output",
required=True,
type=str,
help="Output directory.")
parser.add_argument("-f", "--format",
required=False,
type=str,
default='rgba',
help="Format, one of [grey|rgb] as default is rgba")
version_string = __version__
friendly_version_string = version_string if version_string else 'unknown'
parser.add_argument(
"--version",
action='version',
version='labelbox_export_json version ' + friendly_version_string)
args = parser.parse_args(args)
run_labelbox_json_export(args.json,
args.output,
args.format)
# coding=utf-8
"""Command line processing"""
import argparse
from labelboxutils import __version__
from labelboxutils.ui.labelboxutils_demo import run_demo
def main(args=None):
"""Entry point for Labelbox Utils application"""
parser = argparse.ArgumentParser(description='Labelbox Utils')
## ADD POSITIONAL ARGUMENTS
parser.add_argument("x",
type=int,
help="1st number")
parser.add_argument("y",
type=int,
help="2nd number")
# ADD OPTINAL ARGUMENTS
parser.add_argument("-m", "--multiply",
action="store_true",
help="Enable multiplication of inputs."
)
parser.add_argument("-v", "--verbose",
action="store_true",
help="Enable verbose output",
)
version_string = __version__
friendly_version_string = version_string if version_string else 'unknown'
parser.add_argument(
"--version",
action='version',
version='Labelbox Utils version ' + friendly_version_string)
args = parser.parse_args(args)
run_demo(args.x, args.y, args.multiply, args.verbose)
# coding=utf-8
"""Hello world demo module"""
from labelboxutils.algorithms import addition, multiplication
def run_demo(input_x, input_y, multiply, verbose):
""" Run the application """
if multiply:
result = multiplication.multiply_two_numbers(input_x, input_y)
else:
result = addition.add_two_numbers(input_x, input_y)
if verbose:
if multiply:
print("Calculating {} * {}".format(input_x, input_y))
else:
print("Calculating {} + {}".format(input_x, input_y))
print("Result is {}".format(result))
return result
......@@ -2,6 +2,8 @@
# It is used by pip to manage software dependencies. It is not to be
# confused with the software requirements, which are listed in
# doc/requirements.rst
six
numpy
ipykernel
nbsphinx
opencv-contrib-python
......@@ -19,7 +19,7 @@ setup(
long_description_content_type='text/x-rst',
url='https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils',
author='Matt Clarkson',
author_email='YOUR-EMAIL@ucl.ac.uk',
author_email='m.clarkson@ucl.ac.uk',
license='Apache Software License 2.0',
classifiers=[
'Development Status :: 3 - Alpha',
......@@ -29,9 +29,7 @@ setup(
'Intended Audience :: Information Technology',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
......@@ -51,13 +49,16 @@ setup(
),
install_requires=[
'six>=1.10',
'numpy>=1.11',
'six',
'numpy',
'ipykernel',
'nbsphinx',
'opencv-contrib-python'
],
entry_points={
'console_scripts': [
'labelboxutils=labelboxutils.__main__:main',
'labelbox_export_json=labelboxutils.ui.labelbox_export_json_command_line:main',
],
},
)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment