Issue #1: Implement a script to export masks from Labelbox.

.gitlab-ci.yml

@@ -80,42 +80,6 @@ test Windows:
   tags:
     - shared-win
 
-# Template for deploy stages that depend on 'build docs'
-# if the build stage is skipped, skip these also
-.deploy-docs-skip-template: &deploy-docs-skip-template
-  stage: deploy
-  when: manual
-  except:
-    variables:
-      - $CI_COMMIT_MESSAGE =~ /\[skip[ _-]build?\]/i
-  dependencies:
-    - build docs
-
-deploy docs to staging:
-  <<: *deploy-docs-skip-template
-  script:
-    # Note: the group/username directory must already exist on the server before calling this command
-    - rsync -avz -e'ssh -v' --numeric-ids --delete doc/build/html/* staging_docs_rsync:WEISS/SoftwareRepositories/LabelboxUtils 2>&1
-  tags:
-    - docs-staging
-  environment:
-    name: staging
-    url: http://weisslab-lin.cs.ucl.ac.uk/staging/WEISS/SoftwareRepositories/LabelboxUtils
-  only:
-    - master
-
-deploy docs to production:
-  <<: *deploy-docs-skip-template
-  script:
-    # Note: the group/username directory must already exist on the server before calling this command
-    - rsync -avz -e'ssh -v' --numeric-ids --delete doc/build/html/* production_docs_rsync:WEISS/SoftwareRepositories/LabelboxUtils 2>&1
-  tags:
-    - docs-production
-  environment:
-    name: production
-  only:
-    - public
-
 deploy pip to PyPI:
   stage: deploy
   when: manual

README.rst

 Labelbox Utils
-===============================
+==============
 
 .. image:: https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils/raw/master/project-icon.png
    :height: 128px
@@ -20,21 +20,15 @@ Labelbox Utils
     :alt: Documentation Status
 
 
-
 Author: Matt Clarkson
 
-Labelbox Utils is part of the `SNAPPY`_ software project, developed at the `Wellcome EPSRC Centre for Interventional and Surgical Sciences`_, part of `University College London (UCL)`_.
-
-Labelbox Utils supports Python 2.7 and Python 3.6.
+LabelboxUtils contains python scripts for working with Labelbox. Features so
+far include:
 
-Labelbox Utils is currently a demo project, which will add/multiply two numbers. Example usage:
+* ```labelbox_export_json```: Takes .json file and extracts masks.
 
-::
+For command line apps, run with ```-h``` to see more usage information.
 
-    python labelboxutils.py 5 8
-    python labelboxutils.py 3 6 --multiply
-
-Please explore the project structure, and implement your own functionality.
 
 Developing
 ----------
@@ -51,22 +45,16 @@ You can clone the repository using the following command:
 
 Running tests
 ^^^^^^^^^^^^^
-Pytest is used for running unit tests:
-::
 
-    pip install pytest
-    python -m pytest
+Pytest is used for running unit tests, but you should run using tox,
+as per the `PythonTemplate`_ instructions.
 
 
 Linting
 ^^^^^^^
 
-This code conforms to the PEP8 standard. Pylint can be used to analyse the code:
-
-::
-
-    pip install pylint
-    pylint --rcfile=tests/pylintrc labelboxutils
+This code conforms to the PEP8 standard. Pylint is used to analyse the code.
+Again, follow the `PythonTemplate`_ instructions and run via tox.
 
 
 Installing
@@ -79,15 +67,14 @@ You can pip install directly from the repository as follows:
     pip install git+https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils
 
 
-
 Contributing
-^^^^^^^^^^^^
+------------
 
 Please see the `contributing guidelines`_.
 
 
 Useful links
-^^^^^^^^^^^^
+------------
 
 * `Source code repository`_
 * `Documentation`_
@@ -109,7 +96,6 @@ Supported by `Wellcome`_ and `EPSRC`_.
 .. _`Wellcome EPSRC Centre for Interventional and Surgical Sciences`: http://www.ucl.ac.uk/weiss
 .. _`source code repository`: https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils
 .. _`Documentation`: https://LabelboxUtils.readthedocs.io
-.. _`SNAPPY`: https://weisslab.cs.ucl.ac.uk/WEISS/PlatformManagement/SNAPPY/wikis/home
 .. _`University College London (UCL)`: http://www.ucl.ac.uk/
 .. _`Wellcome`: https://wellcome.ac.uk/
 .. _`EPSRC`: https://www.epsrc.ac.uk/

doc/index.rst

@@ -5,9 +5,7 @@ Contents
 .. toctree::
   :maxdepth: 1
 
-  requirements
   Module Contents <modules>
-  notebooks/first_notebook
 
 * :ref:`modindex`
 * :ref:`genindex`

doc/requirements.rst

-.. highlight:: shell
-
-.. _requirements:
-
-===============================================
-Requirements for Labelbox Utils
-===============================================
-
-This is the software requirements file for Labelbox Utils, part of the
-SNAPPY project. The requirements listed below should define
-what Labelbox Utils does. Each requirement can be matched to a unit test that
-checks whether the requirement is met.
-
-Requirements
-~~~~~~~~~~~~
-+------------+--------------------------------------------------------+-------------------------------------+
-|    ID      |  Description                                           |  Test                               |
-+============+========================================================+=====================================+
-|    0000    |  Module has a help page                                |  pylint, see                        |
-|            |                                                        |  tests/pylint.rc and tox.ini        |
-+------------+--------------------------------------------------------+-------------------------------------+
-|    0001    |  Functions are documented                              |  pylint, see                        |
-|            |                                                        |  tests/pylint.rc and tox.ini        |
-+------------+--------------------------------------------------------+-------------------------------------+
-|    0002    |  Package has a version number                          |  No test yet, handled by git.       |
-+------------+--------------------------------------------------------+-------------------------------------+
-
-
-
-

labelboxutils.py → labelbox_export_json.py

@@ -2,7 +2,7 @@
 #  -*- coding: utf-8 -*-
 import sys
 
-from labelboxutils.ui.labelboxutils_command_line import main
+from labelboxutils.ui.labelbox_export_json_command_line import main
 
 if __name__ == "__main__":
     sys.exit(main(sys.argv[1:]))

labelboxutils/__main__.py

-#!/usr/bin/python
-#  -*- coding: utf-8 -*-
-"""LabelboxUtils"""
-
-import sys
-
-if __name__ == "__main__" and not __package__:
-    # To allow the package's main function to be executed without the -m switch,
-    # i.e. "python labelboxutils", we have to explicitly set the
-    # module name and append the parent directory to the sys.path (see PEP 366)
-    from os import path
-    __package__ = "labelboxutils"  # pylint: disable=redefined-builtin
-    sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
-    sys.path.append(path.dirname(path.dirname(__file__)))
-
-# pylint: disable=wrong-import-position
-from labelboxutils.ui.labelboxutils_command_line import main
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))

labelboxutils/algorithms/addition.py

-# coding=utf-8
-""" Module for adding numbers. """
-
-def add_two_numbers(input_x, input_y):
-    """ Add two numbers """
-    return input_x + input_y

labelboxutils/algorithms/multiplication.py

-# coding=utf-8
-""" Module for multiplying numbers. """
-
-def multiply_two_numbers(input_x, input_y):
-    """ Multiply two numbers """
-    return input_x * input_y

labelboxutils/ui/labelbox_export_json_app.py

+# coding=utf-8
+
+""" Processing for labelbox_export_json appliction. """
+
+import os
+import json
+import urllib.request
+import six
+import cv2
+
+
+def run_labelbox_json_export(json_file, output_dir, output_format):
+    """
+    Parses the Labelbox json file, and exports data.
+
+    :param json_file: file name of json file
+    :param output_dir: where to write to
+    :param output_format: output image format, [grey|rgb|rgba (default)]
+    """
+    if not os.path.exists(output_dir):
+        os.makedirs(output_dir)
+
+    with open(json_file) as file:
+        data = file.read()
+        decoded_data = json.loads(data)
+        for sample in decoded_data:
+            dataset_name = sample['Dataset Name']
+            external_id = sample['External ID']
+            basename = os.path.basename(external_id)
+            filename_no_extension = os.path.splitext(basename)[0]
+            objects = sample['Label']['objects']
+
+            dir_name = os.path.join(output_dir, dataset_name)
+            if not os.path.exists(dir_name):
+                os.makedirs(dir_name)
+
+            for obj in objects:
+                instance_uri = obj['instanceURI']
+                value = obj['value']
+                file_name = os.path.join(dir_name,
+                                         filename_no_extension
+                                         + '_' + value + '.bmp')
+                six.print_("writing to:" + file_name)
+
+                urllib.request.urlretrieve(instance_uri, file_name)
+
+                if output_format == 'grey':
+                    image = cv2.imread(file_name)
+                    grey = cv2.cvtColor(image, cv2.COLOR_BGRA2GRAY)
+                    file_name = os.path.join(dir_name,
+                                             filename_no_extension
+                                             + '_' + value + '.png')
+                    cv2.imwrite(file_name, grey)
+                    six.print_("writing to:" + file_name)
+
+                elif output_format == 'rgb':
+                    image = cv2.imread(file_name)
+                    rgb = cv2.cvtColor(image, cv2.COLOR_BGRA2BGR)
+                    file_name = os.path.join(dir_name,
+                                             filename_no_extension
+                                             + '_' + value + '.png')
+                    cv2.imwrite(file_name, rgb)
+                    six.print_("writing to:" + file_name)
+

labelboxutils/ui/labelbox_export_json_command_line.py

+# coding=utf-8
+
+""" Command line processing for labelbox_export_json. """
+
+import argparse
+from labelboxutils import __version__
+from labelboxutils.ui.labelbox_export_json_app import run_labelbox_json_export
+
+
+def main(args=None):
+    """ Entry point for labelbox_export_json application. """
+    parser = argparse.ArgumentParser(description='LabelBox JSON Export')
+    parser.add_argument("-j", "--json",
+                        required=True,
+                        type=str,
+                        help="JSON file exported from Labelbox.")
+
+    parser.add_argument("-o", "--output",
+                        required=True,
+                        type=str,
+                        help="Output directory.")
+
+    parser.add_argument("-f", "--format",
+                        required=False,
+                        type=str,
+                        default='rgba',
+                        help="Format, one of [grey|rgb] as default is rgba")
+
+    version_string = __version__
+    friendly_version_string = version_string if version_string else 'unknown'
+    parser.add_argument(
+        "--version",
+        action='version',
+        version='labelbox_export_json version ' + friendly_version_string)
+
+    args = parser.parse_args(args)
+
+    run_labelbox_json_export(args.json,
+                             args.output,
+                             args.format)

labelboxutils/ui/labelboxutils_command_line.py

-# coding=utf-8
-
-"""Command line processing"""
-
-
-import argparse
-from labelboxutils import __version__
-from labelboxutils.ui.labelboxutils_demo import run_demo
-
-
-def main(args=None):
-    """Entry point for Labelbox Utils application"""
-
-    parser = argparse.ArgumentParser(description='Labelbox Utils')
-
-    ## ADD POSITIONAL ARGUMENTS
-    parser.add_argument("x",
-                        type=int,
-                        help="1st number")
-
-    parser.add_argument("y",
-                        type=int,
-                        help="2nd number")
-
-    # ADD OPTINAL ARGUMENTS
-    parser.add_argument("-m", "--multiply",
-                        action="store_true",
-                        help="Enable multiplication of inputs."
-                        )
-
-    parser.add_argument("-v", "--verbose",
-                        action="store_true",
-                        help="Enable verbose output",
-                        )
-
-    version_string = __version__
-    friendly_version_string = version_string if version_string else 'unknown'
-    parser.add_argument(
-        "--version",
-        action='version',
-        version='Labelbox Utils version ' + friendly_version_string)
-
-    args = parser.parse_args(args)
-
-    run_demo(args.x, args.y, args.multiply, args.verbose)

labelboxutils/ui/labelboxutils_demo.py

-# coding=utf-8
-
-"""Hello world demo module"""
-from labelboxutils.algorithms import addition, multiplication
-
-def run_demo(input_x, input_y, multiply, verbose):
-    """ Run the application """
-
-    if multiply:
-        result = multiplication.multiply_two_numbers(input_x, input_y)
-
-    else:
-        result = addition.add_two_numbers(input_x, input_y)
-
-
-    if verbose:
-        if multiply:
-            print("Calculating {} * {}".format(input_x, input_y))
-
-        else:
-            print("Calculating {} + {}".format(input_x, input_y))
-
-    print("Result is {}".format(result))
-
-    return result

requirements.txt

@@ -2,6 +2,8 @@
 # It is used by pip to manage software dependencies. It is not to be
 # confused with the software requirements, which are listed in
 # doc/requirements.rst
+six
 numpy
 ipykernel
 nbsphinx
+opencv-contrib-python

setup.py

@@ -19,7 +19,7 @@ setup(
     long_description_content_type='text/x-rst',
     url='https://weisslab.cs.ucl.ac.uk/WEISS/SoftwareRepositories/LabelboxUtils',
     author='Matt Clarkson',
-    author_email='YOUR-EMAIL@ucl.ac.uk',
+    author_email='m.clarkson@ucl.ac.uk',
     license='Apache Software License 2.0',
     classifiers=[
         'Development Status :: 3 - Alpha',
@@ -29,9 +29,7 @@ setup(
         'Intended Audience :: Information Technology',
         'Intended Audience :: Science/Research',
 
-
-       'License :: OSI Approved :: Apache Software License',
-
+        'License :: OSI Approved :: Apache Software License',
 
         'Programming Language :: Python',
         'Programming Language :: Python :: 2',
@@ -51,13 +49,16 @@ setup(
     ),
 
     install_requires=[
-        'six>=1.10',
-        'numpy>=1.11',
+        'six',
+        'numpy',
+        'ipykernel',
+        'nbsphinx',
+        'opencv-contrib-python'
     ],
 
     entry_points={
         'console_scripts': [
-            'labelboxutils=labelboxutils.__main__:main',
+            'labelbox_export_json=labelboxutils.ui.labelbox_export_json_command_line:main',
         ],
     },
 )