Compas_dev_env script

compas
1

hi,

I wrote a little pyinvoke script to create a compas dev env, with the objective of paving the way to easily build a coherent development environment, which can be involved.

functionality

  • builds a conda env with binary modules preinstalled
  • git clones’ the source code of published compas repo’s
    • optionally creates fork’d repo’s such to facilitate pull requests
    • runs a pip install -e . which keep sources in sync with your site-packages via a symbolic link
    • running pip install is interesting, since it’ll make explicit how the various compas_* modules are interlinked. For example compas_fab requires compas>=0.11,<0.14 due to upstream changes in Mesh see issue #128. It’s relevant to see compas ( from master ) being uninstalled here, and good to see compas_fab having pinned dependencies
  • finds compas_* repo’s and performs a pull
  • run pytests in all compas_* modules

why?

In short, this helps to live on the bleeding edge :hocho: a little easier.

Installing software is an annoying barrier to entry, if there’s a clear way how the install is put together, that helps…

Its useful to see how the different repos are interlinked.
If changes are made to compas it’ll propagate to repos with this dependency, and thus its useful to see whether libs that depend on compas are affected by changes.

as an example: compas_tna doesn’t specify a compas versions in requirements.txt, so its easy to have mistakes such as:

  File "/Users/jelleferinga/Code/CADCAM/compas_tna/src/compas_tna/diagrams/formdiagram.py", line 17, in <module>
    from compas.datastructures import network_find_cycles
ImportError: cannot import name 'network_find_cycles' from 'compas.datastructures' (/Users/jelleferinga/miniconda3/envs/compas/lib/python3.8/site-packages/compas/datastructures/__init__.py)

Two libs don’t install correct via this script as of yet:

  • compas_libigl ( libigl only available for Linux )
  • conda_loadpath ( missing setup.py )
1 Like
2

That sounds good. Is the setup script open-sourced?

3

Wow! This is awesome!! :clap:

@tomvanmele: could you help out with the compas_libigl install?

Yep, @jelle linked the github repo in his post, but here it is again: GitHub - jf---/compas_dev_env: create a development environment for the compas eco system

1 Like
4

Thanks, @gonzalocasas!

I believe the current installation config for compas_libigl relies on a local c++ build through cmake. Not sure how that would play out here. There’s an alpha-version python binding of libigl for conda though: conda install -c conda-forge igl

More info: https://github.com/libigl/libigl-python-bindings

5

i can try, but i would need to understand the problem better. the pip installation should work everywhere the same(ish)…

6

Thx so much @gonzalocasas

ah, I search for libigl, great that helps, added with meshplot ( looks very cool… )

Building a proper development environment takes too much time.
The odds of a combinatoric explosion are practically a given, when there is no singular / reproducible way of installing software.

Basically, if there is no deterministic way of installation, there’s gonna be tons of installation related issues and that’ll affect the pleasure of developing the software and eat away at momentum.

Conda and pip are also rather different.
Conda is much more of a ( platform agnostic ) package manager ( apt-get, choco, brew ), where pip was conceived to install pure python. Moreover pip is more of a build tool ( compiling from source code ), whereas conda installs binaries.

Since wheels have come fwd it has become possible to install pre-compiled libs, but its by far not as convincing as conda (the packages available are much more limited) and since the best way to build wheels is via conda-wheels. Conda has about half a decade and hence is quite mature which implies a vast ecosystem of modules & libs.

Conda also does a much better job at resolving dependencies.
Say compas_tna requires compas v0.12 where compas_pattern requires v0.15. Pip will -annoyingly- install 0.12 uninstall 0.12 and install 0.15, dragging you in the tarpool. The point in case - life is sad without a SAT solver handling your dependencies.

Recently conda installs ROS ( across platforms! ) whereas notoriously, ROS is not only tied to a platform, Linux, but even a distro ( Ubuntu )

# create a new environmnet named ros
conda create --name ros --channel conda-forge
   ros-core \
   ros-actionlib \
   ros-dynamic-reconfigure
   python=X.X  # select your desired Python version here (2.7, 3.6, 3.7)
conda activate ros
...
now you can use any ros command from the core set, such as 
roscore
roslaunch ... 
rostopic ...
rosnode ...

( gossip: perhaps conda will be further integrated in ROS )

conda even allows to install FreeCAD:

 conda install -c conda-forge freecad

The point in case is that conda is a great way to install development environments.

This turns out to be pretty key.
Say you want to update a lib to an updated dependency, say a new release of conda-libigl, or change from pyqt → pyside2, or upgrade from vtk 7 → vtk 8

Rather than uninstalling libigl installing the newly released version, just build an additional environment with this script and use that env.

This examples applies to testing on various [C]Python interpreters.

You w(ish), its not very rigour(ish) and certainly not deterministic(ish) :wink:

Sorry - all kiddin’ aside, both pip and conda are used. Pip is used to install the compas_* packages, via pip install -e . which creates a symbolic link, such that changes in the code are reflected. Conda is used to install the libs and more complex libs and install the required development tools ( llvm compiler for numba, cmake ) and modules that are not pip installable and or depend on external libs ( igl, planarity, pillow )

The point in case is that the suggested approach is to facilitate a development focussed install, which is complimentary to the setup.py / pip approach.

Anyway.
Sorry for the exhaustive reply.
Point in case is that sometimes some properly tooling might help to focus on the domain problem

1 Like
7

i was just asking what you meant with “libigl only available on linux”…

8

First I couldnt find the conda package for libigl, but its been added incl meshplot

9

i am aware of the conda package but compas_libigl isn’t built upon it because at the time of starting compas_libigl it was not yet available. and afaik, the functions i was trying to use are still not available through the python bindings on conda.

in any case,

compas_libigl will clearly be obsolete at some point in the near future. i just keep it around to explore making bindings for C++ libs with PyBind and using those in, for example, Rhino…

(it was also used as a dependency for RhinoVAULT2 until recently, but we have replaced it there with compas_triangle)

1 Like
10

Is RhinoVAULT2 == compas_tna?

11

no, since RhinoVAULT doesn’t work anymore in the newer releases of Rhino (basically anything above 5), we are working on a new version that is entirely based on COMPAS, using compas_cloud, compas_ags, compas_tna, compas_pattern, compas_skeleton, compas_triangle, and that can thus be used in Rhino and in Blender.

the core setup, the installer, and the Rhino UI are almost ready and currently being tested internally at BRG.

the first beta release should be available around easter…

1 Like
12

13

Just a tiny bit of info: Instead of manually crafting the base environment.yml, one option would be to do conda install --only-deps compas. This installs all dependencies of compas (allowing even to specify version and get the deps accordingly matched from conda-forge stuff), but does not install compas itself.

After that, compas and other packages can be installed with pip install -e . as you have in the scripts.

1 Like