Introducing the TDPyEnvManager COMP
TDPyEnvManager COMP is designed to become your homebase when working with sideloaded Python environments and third party libraries. We aim at making it the trusted go-to solution for TouchDesigner users and advanced developers.TDPyEnvManager COMP is a custom component that helps manage Python environments (vEnv) as well as Conda environments, the primary focus is for sideloading environments and integrating third party libraries more easily in TouchDesigner.!!! Sideloading Python environments and third party packages is at your own risk and can cause instabilities. For more details, head to https://docs.derivative.ca/Python
vEnvs and conda environments from the component's parameter dialog. Users can interact directly with pip in both vEnvs and conda CLIs, as well as conda commands when using conda.!!! With the complexity of covering multiple systems (ie. Windows, MacOS, and environments from Python and Conda), comes the complexity of testing. Some edge cases might not be covered and we are inviting the community for feedback. If you find that you are missing a feature, or that an implemented feature might need a change or some UX improvements, feel free to send us a message in the forum.
Using TDPyEnvManager with Python vEnv
TDPyEnvManager. While we could support management of multiple vEnvs, the idea here is to enable for only one vEnv per project folder with a limited set of features for a common and streamlined approach.Let’s get started.
MyTDEnvTest.MyTDEnvTest a .toe file named MyTDEnvTest.toe. Then open this new .toe file. Delete any OPs in /project1.Active toggle. This toggle is handy when you have already setup an environment but you want to prevent it from being added to the TouchDesigner search path on startup for debugging purposes or prototyping.You’ll see a dialog with a disclaimer stating that sideloading environments and using third party libraries in TouchDesigner is at your own risk. You can click continue if you are fine with the disclaimer.
Python vEnv to be streamlined and straightforward, you can simply click on Create vEnv to get going.vEnv will be following the same naming schemes on all machines. It will use the project folder name and add the suffix _vEnv after it.MyTDEnvTest_vEnv will be created in MyTDEnvTest next to MyTDEnvTest.toe.TDPyEnvManager, as well as the Status parameter in the parameter dialog updating.vEnv will be done and you should see in the Status the message Environment linked and ready.Open CLI: it will open a new terminal, with the vEnv you just created being activated.pip install zmq. It will look for the zmq package on PyPi and download and install it in the vEnv.Alt+T), type import zmq followed by zmq. It will import zmq and show in the textport the zmq module as well as where it’s loaded from.zmq was loaded from wherever you created your MyTDEnvTest folder: MyTDEnvTest/MyTDEnvTest_vEnv/Lib/site-packages/zmq/__init__.py! A note about UV - If trendy UV is your preferred tool, you can install it within yourvEnvusingpip install uvYou can then useuv your_commandin the running CLI.
Sharing my vEnv setup with others
Export requirements.txtrequirements.txt file next to your currently running .toe file.!!! Important note: This is not an optimalrequirements.txtfile. You need to go through it manually and curate it to avoid setting too many version restrictions and possibly add a--extra-index-url <https://some.url>line within therequirements.txtfile.
requirements.txt with another user.requirements.txt file next to the .toe file they have open, and when they click on Create from requirements.txt it will create a vEnv and install all the required packages, such as zmq.! If a matching folder of the same name is already present, the folder doesn’t get cleared. The environment gets created if a file is missing in the target folder, but an existing environment will be unaffected unless there is a change in dependencies caused by the Python builtins or a possiblerequirements.txtfile being present. This is valid for both creating a barebonevEnvfrom TouchDesigner or creating avEnvwith arequirements.txtfile being present.
Using TDPyEnvManager with Conda
Python vEnv approach. We are using conda and the Miniconda installer. On top of allowing for conda packages to be installed (as well as pip packages), we can specify the conda install folder in which environments will be looked for and created, we can create multiple conda environments, and list conda environments that are installed on the machine globally. Similar to the Python vEnv approach, we can export our environment.yml files and create environments from environment.yml files.!!! When using thecondamode, TouchDesigner is downloading and installing the Miniconda installer in the background. When silently installing the Miniconda installer, users agree to the Anaconda Inc. ToS: https://legal.anaconda.com/policies/en/TouchDesigner adds restrictions so that environments created are setup to fetch packages from conda-forge channels, which are not the default channels covered by the Anaconda Inc. ToS. However, there is a probability for user or system configurations to take priority and the restrictions might not always apply.
Creating and activating a conda environment
.toe file.TDPyEnvManagerActive parameter on, and click Continue in the popup with the disclaimer.Conda Env in the Mode parameter.Conda Install Folder parameter as is.MySuperCondaEnv.Create Conda Env..toe file. Once it’s downloaded, it will start the installation.! If you are working offline and already downloaded the Miniconda installer, place it next to your.toefile and click on the toggleKeep Conda Installer. It will re-use the installer and keep it after install. On Windows, you can find the installer at the following link: https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe. Keep it namedMiniconda3-latest-Windows-x86_64.exe. On MacOS, you can find the installer at the following link: https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh. Keep it namedMiniconda3-latest-MacOSX-arm64.sh.
Continue to proceed.conda will proceed and create a new environment.Now, you can proceed and click on the Open CLI pulse parameter to open the conda prompt with your activated environment.
MySuperCondaEnv, I see the CLI input being preceded by the (MySuperCondaEnv) which indicates which environment is activated.conda commands, such as conda install as well as pip commands.tqdm with conda install tqdm.import tqdm followed by tqdm and see that my package is properly installed and imported from my MySuperCondaEnv environment.Listing global environments
conda install, toggling on the Include Global parameter will list all environments on the machine with a Python version matching TouchDesigner’s. Global environments are in the dropdown menu of the Environment parameter and are suffixed by a star *.Sharing my conda env setup with others
Export environment.yml and an environment.yml will be created, relative to your .toe file. The difference with a Python requirements.txt file is that it includes the Python version as a requirement to the environment created, the environment name, as well as packages of various sources and the channels in use.environment.yml file next to the .toe file they are currently using and click on the Create from environment.yml pulse parameter.!!! Note that if a matching environment already exists, it will be overridden by the environment.yml requirements.
Advanced usage
TDPyEnvManager programmatically using its extension promoted attributes and methods, or you can use the TDPyEnvManagerHelper directly. The TDPyEnvManagerHelper can be used as a module and imported from the TDPyEnvManager COMP/TDPyEnvManagerHelper or as an attribute of the TDPyEnvManager itself, Helper.Samples/TDPyEnvManager folder for samples.Considerations
TDPyEnvManager in your projects, and when custom components are relying on third party libraries loaded from side loaded environments, it is preferable that you let the initialization of custom components being handled lazily.Init On Start toggle.TDPyEnvManager should be the first component initializing on start so that the environment is setup before any additional component attempt to import modules from it.try / except condition around your import statements at the top of your extension.try: import tensorrt as trt import torch import torchvision.transforms as transforms except Exception as e: logger.error(f'TDDepthAnythingHF - An error occured trying to import some of the required libraries. Make sure that the environment is setup properly.') logger.error(f'TDDepthAnythingHF - {e}')
When setting up your TDPyEnvManager and side environment, you can safely ignore issues from custom components. When your environment is setup properly and you installed the required dependencies, restart TouchDesigner and any import issue should be fixed.
Tips
{ "folders": [ { "path": "." } ], "settings": { "python.defaultInterpreterPath": "C:\\Program Files\\Derivative\\TouchDesigner.2023.32465\\bin\\python.exe", "python.analysis.autoSearchPaths": true, "python.autoComplete.extraPaths": [ "C:\\Program Files\\Derivative\\TouchDesigner.2023.32465\\bin\\python\\Lib\\site-packages", ".\\scripts", ".\\TDDepthAnythingHF_vEnv\\Lib\\site-packages", ], "python.analysis.extraPaths": [ "C:\\Program Files\\Derivative\\TouchDesigner.2023.32465\\bin\\python\\Lib\\site-packages", ".\\scripts", ".\\TDDepthAnythingHF_vEnv\\Lib\\site-packages", ], } }
In the example above, the interpreter is pointing to TouchDesigner’s own Python binary and we added TouchDesigner site-packages as well as a Python vEnv plus a relative scripts folder.
Conclusion
TDPyEnvManager and the Thread Manager to work with the Hugging Face transformers library and Depth Anything V2.





