Cat-Net: Canonical Appearance Transformations

CAT-Net: Learning Canonical Appearance Transformations

Code to accompany our paper "How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change".


  • numpy
  • matpotlib
  • pytorch + torchvision (1.2)
  • Pillow
  • progress (for progress bars in train/val/test loops)
  • tensorboard + tensorboardX (for visualization)
  • pyslam + liegroups (optional, for running odometry/localization experiments)
  • OpenCV (optional, for running odometry/localization experiments)

Training the CAT

  1. Download the ETHL dataset from here or the Virtual KITTI dataset from here
    1. ETHL only: rename ethl1/2 to ethl1/2_static.
    2. ETHL only: Update the local paths in tools/ and run python3 tools/ to generate a synchronized copy of the real sequences.
  2. Update the local paths in run_cat_ethl/ and run python3 run_cat_ethl/ to start training.
  3. In another terminal run tensorboard --port [port] --logdir [path] to start the visualization server, where [port] should be replaced by a numeric value (e.g., 60006) and [path] should be replaced by your local results directory.
  4. Tune in to localhost:[port] and watch the action.

Running the localization experiments

  1. Ensure the pyslam and liegroups packages are installed.
  2. Update the local paths in and run python3 [dataset] to compile the model outputs into a localization_data directory.
  3. Update the local paths in run_localization_[dataset].py and run python3 run_localization_[dataset].py [rgb,cat] to compute VO and localization results using either the original RGB or CAT-transformed images.
  4. You can compute localization errors against ground truth using the script, which generates CSV files and several plots. Update the local paths and run python3 [dataset].


If you use this code in your research, please cite:

  author = {Lee Clement and Jonathan Kelly},
  journal = {{IEEE} Robotics and Automation Letters},
  link = {},
  title = {How to Train a {CAT}: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change},
  year = {2018}

Download Details:

Author: utiasSTARS
Source Code: 
License: MIT license

#machinelearning #python #pytorch #deeplearning #computervision 

Cat-Net: Canonical Appearance Transformations
1.00 GEEK