Skip to content

Conda-build recipes

To enable building conda packages, install and update conda and conda-build.

Building a conda package requires a recipe. A conda-build recipe is a flat directory that contains the following files:

  • meta.yaml - A file that contains all the metadata in the recipe. Only package/name and package/version are required.
  • build.sh - The script that installs the files for the package on macOS and Linux. It is executed using the bash command.
  • bld.bat - The build script that installs the files for the package on Windows. It is executed using cmd.
  • run_test.[py,pl,sh,bat] - An optional Python test file, a test script that runs automatically if it is part of the recipe.
  • Optional patches that are applied to the source.
  • Other resources that are not included in the source and cannot be generated by the build scripts. Examples are icon files, readme files and build notes.

Tip

When you use the conda skeleton skeleton_ref command, the first 3 files, meta.yaml, build.sh, and bld.bat, are automatically generated for you.

Conda-build process

conda-build performs the following steps:

  1. Reads the metadata.
  2. Downloads the source into a cache.
  3. Extracts the source into the source directory.
  4. Applies any patches.
  5. Re-evaluates the metadata, if source is necessary to fill any metadata values.
  6. Creates a build environment and then installs the build dependencies there.
  7. Runs the build script. The current working directory is the source directory with environment variables set. The build script installs into the build environment.
  8. Performs some necessary post-processing steps, such as shebang and rpath.
  9. Creates a conda package containing all the files in the build environment that are new from step 5, along with the necessary conda package metadata.
  10. Tests the new conda package if the recipe includes tests:
    1. Deletes the build environment.
    2. Creates a test environment with the package and its dependencies.
    3. Runs the test scripts.

The conda-recipes repo contains example recipes for many conda packages.

Caution

All recipe files, including meta.yaml and build scripts, are included in the final package archive that is distributed to users. Be careful not to put sensitive information such as passwords into recipes where it could be made public.

The conda skeleton skeleton_ref command can help make skeleton recipes for common repositories, such as PyPI.

Deep dive

Let's take a closer look at how conda-build uses a recipe to create a package.

Templates

When you build a conda package, conda-build renders the package by reading a template in the meta.yaml. See jinja-templates.

Templates are filled in using your conda-build config, which shows the matrix of things to build against. The conda build config determines how many builds it has to do. For example, defining a conda_build_config.yaml of the form and filling it defines a matrix of 4 packages to build:

foo:
    - 1.0
    - 2.0
bar:
    - 1.2.0
    - 1.4.0

After this, conda-build determines what the outputs will be. For example, if your conda build config indicates that you want two different versions of Python, conda-build will show you the rendering for each Python version.

Environments

To build the package, conda-build will make an environment for you and install all of the build and run dependencies in that environment. Conda-build will indicate where you can successfully build the package. The prefix will take the form:

<path to conda>/conda-bld/<package name and string>/h_env_placeholder…

Conda-forge downloads your package source and then builds the conda package in the context of the build environment. For example, you may direct it to download from a Git repo or pull down a tarball from another source.

What conda-build puts into a package depends on what you put into the build, host, or run sections. Conda-build will use this information to identify dependencies to link to and identify the run requirements for the package. This allows conda-build to understand what is needed to install the package.

Building

Once the content is downloaded, conda-build runs the build step. The build step runs a script. It can be one that you provided.

If you do not define the script section, then you can create a build.sh or a bld.bat file to be run.

Prefix replacement

When the build environment is created, it is in a placeholder prefix. When the package is all bundled up, the prefix is set to a dummy prefix. When conda is ready to install the package, it rewrites the dummy prefix with the correct one.

Testing

Once a package is built, conda-build will test it. To do this, it creates another environment and installs the conda package. The form of this prefix is:

<path to conda>/conda-bld/<package name + string>/_test_env_placeholder…

At this point, conda-build has all of the info from the meta.yaml about what its runtime dependencies are, so those dependencies are installed as well. This generates a test runner script with a reference to the testing meta.yaml that is created.

Output metadata

After the package is built and tested, conda-build cleans up the environments created prior and outputs the metadata. The recipe for the package is also added in the output metadata. The metadata directory is on the top level of the tarball in the info directory. The metadata contains information about the dependencies of the package and a list of where all of the files in the package go when it is installed. Conda reads that metadata when it needs to install.

Running conda install causes conda to:

  • reach out to the repo data containing the dependencies,
  • guess the right dependencies,
  • install a list of packages,
  • unpack the tarball to look at the info,
  • verify the file based on metadata in the package, and then
  • go through each file in the package and put it in the right location.