Michio JP

Michio JP

1629711839

Hand-Object Reconstruction from RGB Videos

Towards uncontrained hand-object reconstruction from RGB videos

drawing

Yana Hasson, Gül Varol, Ivan Laptev and Cordelia Schmid

Table of Content

Demo

Open In Colab

Setup

Environment setup

Note that you will need a reasonably recent GPU to run this code.

We recommend using a conda environment:

conda env create -f environment.ymlconda activate phosa16

External Dependencies

Detectron2, NMR, FrankMocap

mkdir -p external
git clone --branch v0.2.1 https://github.com/facebookresearch/detectron2.git external/detectron2
pip install external/detectron2

Install a slightly modified fast version of Neural Mesh Renderer (NMR) (modification makes this implementation of NMR pytorch 1.6 compatible :) )

Install FrankMocap, with a slight twist to return the detected objects from Understanding Human Hands in Contact at Internet Scale, Shan et al., CVPR 2020.

mkdir -p external
git clone https://github.com/hassony2/frankmocap.git external/frankmocap
sh scripts/install_frankmocap.sh

Install MANO

Follow the instructions below to install MANO

  • Go to MANO website: http://mano.is.tue.mpg.de/
  • Create an account by clicking *Sign Up* and provide your information
  • Download Models and Code (the downloaded file should have the format mano_v*_*.zip). Note that all code and data from this download falls under the MANO license (see http://mano.is.tue.mpg.de/license).
  • Unzip and copy the content of the *models* folder into the extra_data/mano folder

Install SMPL-X

Follow the instructions below to install SMPL-X

  • Go to SMPL-X website: https://smpl-x.is.tue.mpg.de/
  • Create an account by clicking *Sign Up* and provide your information
  • Go to SMPL-X download page: https://smpl-x.is.tue.mpg.de/downloads
  • Download the v1.1 model. Note that all data from this download falls under the SMPL-X license (see https://smpl-x.is.tue.mpg.de/license).
  • Unzip and copy the content of the *models* folder into the extra_data/smpl folder

Download datasets

HO-3D

Download the dataset following the instructions on the official project webpage.

This code expects to find the ho3d root folder at local_data/datasets/ho3d

Core50

Follow instructions below to setup the Core50 dataset

  • Download the Object models from ShapeNetCorev2
    • Go to https://shapenet.org and create an account
    • Go to the download ShapeNet page 
    • You will need the "Archive of ShapeNetCore v2 release" (~25GB)
    • unzip to local_data folder by adapting the command
      • unzip /path/to/ShapeNetCore.v2.zip -d local_data/datasets/ShapeNetCore.v2/
  • Download the images from Core50

Running the Code

Check installation

Make sure your file structure after completing all the Setup steps, your file structure in the homan folder looks like this.

# Installed datasets
local_data/
  datasets/
    ho3d/
    core50/
    ShapeNetCore.v2/
    epic/
# Auxiliary data needed to run the code
extra_data/
  # MANO data files
  mano/
    MANO_RIGHT.pkl
    ...
  smpl/
    SMPLX_NEUTRAL.pkl

Start fitting

Core50

Step 1

  • Pre-processing images
  • Joint optimization with coarse interaction terms
python fit_vid_dataset.py --dataset core50 --optimize_object_scale 0 --result_root results/core50/step1

Step 2

  • Joint optimization refinement
python fit_vid_dataset.py --dataset core50 --split test --lw_collision 0.001 --lw_contact 1 --optimize_object_scale 0 --result_root results/core50/step2 --resume results/core50/step1

HO3d

Step 1

  • Pre-processing images
  • Joint optimization with coarse interaction terms
python fit_vid_dataset.py --dataset ho3d --split test --optimize_object_scale 0 --result_root results/ho3d/step1

Step 2

  • Joint optimization refinement
python fit_vid_dataset.py --dataset ho3d --split test --lw_collision 0.001 --lw_contact 1 --optimize_object_scale 0 --result_root results/ho3d/step2 --resume results/ho3d/step1

Acknowledgements

PHOSA

The code for this project is heavily based on and influenced by Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild (PHOSA)] by Jason Y. Zhang*, Sam Pepose*, Hanbyul Joo, Deva Ramanan, Jitendra Malik, and Angjoo Kanazawa, ECCV 2020

Consider citing their work !

@InProceedings{zhang2020phosa,    title = {Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild},    author = {Zhang, Jason Y. and Pepose, Sam and Joo, Hanbyul and Ramanan, Deva and Malik, Jitendra and Kanazawa, Angjoo},    booktitle = {European Conference on Computer Vision (ECCV)},    year = {2020},}

Funding

This work was funded in part by the MSR-Inria joint lab, the French government under management of Agence Nationale de la Recherche as part of the ”Investissements d’avenir” program, reference ANR19-P3IA-0001 (PRAIRIE 3IA Institute) and by Louis Vuitton ENS Chair on Artificial Intelligence.

Other references

If you find this work interesting, you will certainly be also interested in the following publication:

To keep track of recent publications take a look at awesome-hand-pose-estimation by Xinghao Chen.

License

Note that our code depends on other libraries, including SMPL, SMPL-X, MANO which each have their own respective licenses that must also be followed.

mkdir -p external
git clone https://github.com/hassony2/multiperson.git external/multiperson
pip install external/multiperson/neural_renderer
cd external/multiperson/sdf
pip install external/multiperson/sdf

What is GEEK

Buddha Community

Arvel  Parker

Arvel Parker

1591611780

How to Find Ulimit For user on Linux

How can I find the correct ulimit values for a user account or process on Linux systems?

For proper operation, we must ensure that the correct ulimit values set after installing various software. The Linux system provides means of restricting the number of resources that can be used. Limits set for each Linux user account. However, system limits are applied separately to each process that is running for that user too. For example, if certain thresholds are too low, the system might not be able to server web pages using Nginx/Apache or PHP/Python app. System resource limits viewed or set with the NA command. Let us see how to use the ulimit that provides control over the resources available to the shell and processes.

#[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object]

MEAN Stack Tutorial MongoDB ExpressJS AngularJS NodeJS

We are going to build a full stack Todo App using the MEAN (MongoDB, ExpressJS, AngularJS and NodeJS). This is the last part of three-post series tutorial.

MEAN Stack tutorial series:

AngularJS tutorial for beginners (Part I)
Creating RESTful APIs with NodeJS and MongoDB Tutorial (Part II)
MEAN Stack Tutorial: MongoDB, ExpressJS, AngularJS and NodeJS (Part III) 👈 you are here
Before completing the app, let’s cover some background about the this stack. If you rather jump to the hands-on part click here to get started.

#[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object]

Creating RESTful APIs with NodeJS and MongoDB Tutorial

Welcome to this tutorial about RESTful API using Node.js (Express.js) and MongoDB (mongoose)! We are going to learn how to install and use each component individually and then proceed to create a RESTful API.

MEAN Stack tutorial series:

AngularJS tutorial for beginners (Part I)
Creating RESTful APIs with NodeJS and MongoDB Tutorial (Part II) 👈 you are here
MEAN Stack Tutorial: MongoDB, ExpressJS, AngularJS and NodeJS (Part III)

#[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object]

CentOS Linux 8.2 Released and Here is How to Upgrade it

CentOS Linux 8.2 (2004) released. It is a Linux distribution derived from RHEL (Red Hat Enterprise Linux) 8.2 source code. CentOS was created when Red Hat stopped providing RHEL free. CentOS 8.2 gives complete control of its open-source software packages and is fully customized for research needs or for running a high-performance website without the need for license fees. Let us see what’s new in CentOS 8.2 (2004) and how to upgrade existing CentOS 8.1.1199 server to 8.2.2004 using the command line.

#[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object]

Tyrique  Littel

Tyrique Littel

1597723200

FreeBSD s3cmd failed [SSL CERTIFICATE_VERIFY_FAILED]

When I install s3cmd package on my FreeBSD system and try to use the s3cmd command I get the following error:

_ERROR: Test failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (ssl.c:1091)

How do I fix this problem on FreeBSD Unix system?

Amazon Simple Storage Service (s3 ) is object storage through a web service interface or API. You can store all sorts of files. FreeBSD is free and open-source operating systems. s3cmd is a command-line utility for the Unix-like system to upload, download files to AWS S3 service from the command line.

ERROR: Test failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed error and solution

This error indicates that you don’t have packages correctly installed, especially SSL certificates. Let us see how to fix this problem and install s3cmd correctly on FreeBSD to get rid of the problem.

How to install s3cmd on FreeBSD

Search for s3cmd package:

$ pkg search s3cmd

Execute the following command and make sure you install Python 3.x package as Python 2 will be removed after 2020:

$ sudo pkg install py37-s3cmd-2.1.0

Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 8 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	libffi: 3.2.1_3
	py37-dateutil: 2.8.1
	py37-magic: 5.38
	py37-s3cmd: 2.1.0
	py37-setuptools: 44.0.0
	py37-six: 1.14.0
	python37: 3.7.8
	readline: 8.0.4

Number of packages to be installed: 8

The process will require 118 MiB more space.

Proceed with this action? [y/N]: y
[rsnapshot] [1/8] Installing readline-8.0.4...
[rsnapshot] [1/8] Extracting readline-8.0.4: 100%
[rsnapshot] [2/8] Installing libffi-3.2.1_3...
....
..
[rsnapshot] [8/8] Extracting py37-s3cmd-2.1.0: 100%
=====
Message from python37-3.7.8:

--
Note that some standard Python modules are provided as separate ports
as they require additional dependencies. They are available as:

py37-gdbm       databases/py-gdbm@py37
py37-sqlite3    databases/py-sqlite3@py37
py37-tkinter    x11-toolkits/py-tkinter@py37

#[object object] #[object object] #[object object] #[object object] #[object object] #[object object] #[object object]