htrdr The Monte-Carlo radiative transfer simulator

htrdr evaluates the intensity at any position (probe) of the scene, in any direction, in the presence of surfaces and an absorbing and diffusing semi-transparent medium, for both internal (longwave) or external (shortwave) radiation sources. The intensity is calculated using the Monte-Carlo method: a number of optical paths are simulated backward, from the probe position and into the medium. Various algorithms are used, depending on the specificities of the nature and shape of the radiation source.

Applications are theoretically possible to any configuration. However, it all eventually comes down to the possibility of using the physical data of interest, in their most common formats, in each scientific community. htrdr is currently suitable for three main application fields:

1. Atmospheric radiative transfer: a clear-sky atmosphere is vertically stratified, neglecting Earth sphericity, and described in terms of absorption coefficients as a function of height and spectral quadrature point as per a correlated-k model. Cloud physical properties are provided on a 3D rectangular grid. Surface geometrical and optical properties can be provided for an arbitrary number of geometries. Internal radiation and solar radiation are taken into account.

2. Combustion processes: thermodynamic data is provided at the nodes of an unstructured tetrahedral mesh, while surface properties can still be provided for various materials. The radiation source is only external: a monochromatic laser sheet illuminates the inside of the combustion chamber for diagnostic purposes.

3. 

   Images of Titan rendered with htrdr-planeto. Its 3D atmosphere is composed of a gas mixture and two aerosol modes for haze and clouds. In the bottom image, the source is positioned behind Titan to simulate a transit situation, i.e. a solar eclipse by Titan.

   Planetology: takes into account the geometry of a "ground" of arbitrary shape, described by a triangular mesh, with the possibility of using an arbitrary number of materials. The radiative properties of a gas mixture must be provided on a tetrahedral mesh, using the k-distribution spectral model. The radiative properties of an arbitrary number of aerosol and hydrometeores can also be provided on their individual tetrahedral mesh. Calculations can be made for both internal and external radiation sources. In the case of an external source, a sphere of arbitrary size and position is used. This sphere can radiate as a Planck source at a specified brightness temperature, or be associated with a high-resolution radiance spectrum.

Since any radiative transfer observable is expressed as an integral of the radiance, and since there is a strict equivalence between the integral to be solved and the underlying Monte-Carlo algorithm (each integral is associated with the sampling of a random variable), the algorithms that calculate the radiance are used for computing various quantities:

• Images on a camera sensor, in a given field of view. For combustion applications, only monochromatic images are supported. In atmospheres and in planetology, spectral integration is also possible, both for solar and thermal images: CIE colorimetry is used for solar images, while thermal images are in fact brightness temperature maps, obtained from the incoming radiative flux over a specified spectral interval.

• Flux density maps, on a grid of sensors, integrated over an entire hemisphere. In the case of combustion chambers, flux density maps can be calculated, while spectrally integrated flux density maps are also possible for atmospheric application, both for solar and thermal radiation.

Related projects

Images rendered with htrdr-atmosphere of a 1000³ cloud field produced by the Meso-NH (Lafore et al 1998, Lac et al 2018) Large Eddy Model. The initial conditions and model set-up for the simulation are described in Strauss et al. (2019). The infrared rendering is calculated in [9, 10] µm spectral range; the color map displays the brightness temperature in Kelvin.

htrdr has been used, developped and extended in the following research projects:

• The development of htrdr began with the High-Tune project. Originally, it simulated the radiative transfer of an external source (solar) in a scene composed of a triangulated ground and an atmosphere, neglecting Earth sphericity, in the presence of a cloud field provided over a structured grid. It was later extended in order to take into account a non-gray surface, and the possibility to perform radiative transfer computations for a internal source (ground and atmosphere).
• In project ModRadUrb the emphasis was put on taking into account the representation of complex geometries (detailled city scenes) using spectral properties of an arbitrary number of materials. The solver was extended to solve upward and downward hemispherical atmospheric fluxes on a plane positioned anywhere in the scene, both in the visible and the infrared spectral ranges.
• In project MCG-Rad the htrdr codebase was used to explore a whole new class of radiative transfer algorithms: instead of relying on the full atmospheric radiative properties data set (prerequisite for current algorithms), the so-called line sampling algorithms will not require nor compute the absorption coefficient of the atmosphere. Instead, it will sample energetic transitions and use a Line-by-Line parameters database (such as HITRAN) in order to perform a rigorous spectral integration, both in the visible and the infrared spectral ranges.
• In project Astoria, htrdr was used to produce images in the visible, in the presence of combustion chambers where radiation scattering is performed by soot aggregates. One of the main difficulty resides in the fact that the chamber is illuminated by a laser: the classical solar radiative transfer algorithm then fails to converge because of the collimated radiation source, and a new algorithm was thus designed in order to ensure numerical convergence.
• In project Rad-Net, htrdr was adapted for applications in planetology and astrophysics. The application is now a scene composed of an arbitrary number of solid surfaces (a planet, satellites) represented by triangular meshes and materials which describe their spectral reflectivity/emissivity properties. The 3D atmopshere is defined by a number of participating semi-transparent media (a gas mixture and an arbitrary number of aerosol modes) whose radiative properties are provided at the nodes of a unstructured tetrahedral volumetric grid, independant for each medium.

A straight interface

Image rendered with htrdr-atmosphere, of a city procedurally generated with spectral materials defined in particular from data from the Spectral Library of impervious Urban Materials. The entire dataset (geometry, materials and cloud field) is included in the atmosphere Starter-Pack.

htrdr is a command-line tool that performs computations on input data, writes the rendered image and nothing more. No assumption is made on how input data are created, the only requirement is compliance with the expected file formats. In the same spirit, the output image is written in plain text, as a list of raw pixel estimations, which makes it easier to post-process.

This thin interface is, by nature, particularly well suited to be extended and integrated into any workflow. For instance, one can save the cloud properties used by htrdr-atmosphere directly in the htcp file format or use the les2htcp tool to convert cloud properties from NetCDF to htcp. In the same way, the output image can be post-processed through gnuplot or converted in a regular PPM image by the htpp program, and then visualised in an image viewer as for instance feh or display.

Installation

Renderings in the visible range calculated by htrdr-combustion of a laminar sooting flame seen from above and illuminated by a laser sheet. The images display radiation that is emitted by the laser, scattered and transmitted by the combustion medium. The laser sheet is horizontal, and intersects the medium at various heights. One can see the difference in the scattered signal between the left and right parts of the image; since the laser propagates from the left to the right, it is progressively attenuated while traveling the medium. The intensity of radiation subject to scattering therefore decreases. In addition to these trapping effect these images provide some insight about the scattering cross-section of the medium as a function of height. Scattering properties of soot gradually increase from the injection position to a height of approximately 35 mm. A steep decrease follows: the image for a height of 40 mm is very similar to the image obtained at 10 mm.

Images in the visible range rendered with htrdr-combustion of a sooting flame illuminated by an horizontal laser sheet that runs through the central axis of the flame; the camera is looking at the centre of the flame, but its normal is oriented respectively at an angle of 30, 60 and 90 degrees relative to the main direction of the laser sheet. One can see the trapping of radiation by the medium and the decrease of scattering intensity decreases with the angle of observation, otherwise known as the scattering angle and the decrease of the scattered intensity with the angle of observation.

No pre-compiled version of htrdr is provided; it must be compiled directly from its source tree. A simple way is to rely on star-build, which automates the build and installation of htrdr and its dependencies from source code.

Prerequisites

To build htrdr with star-build, first make sure your system has the following prerequisites:

• POSIX shell
• POSIX make
• curl
• git
• mandoc
• pkg-config
• sha512sum
• GNU Compiler Collection in version 8.3 or higher
• netCDF4 library and headers
• OpenMPI library and headers in version 2 or higher

Build

Assuming that the aforementioned prerequisites are available, the build procedure is summed up to:

~ $ git clone \
  https://gitlab.com/meso-star/star-build.git
~ $ cd star-build
~/star-build $ make \
  PREFIX=~/htrdr_0.10 \
  BUILD=src/htrdr_0.10.sh

With PREFIX defining the path where htrdr will be installed and BUILD defining the installation script to be run.

By default, the whole htrdr project is built but you may prefer to deploy htrdr only for a specific application, i.e. only for atmospheric radiative transfer, combustion processes or planetology. For example, to install only the atmospheric part of htrdr:

~/star-build $ make \
  PREFIX=~/htrdr_0.10 \
  BUILD=src/htrdr_0.10.sh \
  ATMOSPHERE=ENABLE \
  COMBUSTION=DISABLE \
  PLANETO=DISABLE

Shortwave (left) and longwave (right) downward flux density maps in W/m² computed by htrdr-atmosphere at 1 meter height with the DZVAR cloud field. The shortwave and longwave spectral integration ranges are [0.38, 4] µm and [4, 100] µm, respectively. Their spatially-avaraged flux is 879.349 W/m² in shortwave and 425.159 W/m² in longwave. In both cases, the sun is located at the zenith. In the shortwave map we observe the contrast between the shadows of the clouds and fully illuminated areas. In longwave, we can see the effect of clouds (higher values, due to the emission by the base of the cloud at higher temperatures than for a clear-sky zone) and also a "ripple" effect that is due to the spatial variations of water vapor concentration, as provided by the LES simulation.

Run

Evaluate the installed htrdr.profile file in the current shell to register htrdr against it. You can then run htrdr and consult its manual pages:

~ $ . ~/htrdr_0.10/etc/htrdr.profile
~ $ htrdr -h
~ $ man htrdr

Refer to the Starter Packs (atmosphere or combustion), to quickly run a htrdr rendering; these archives provide input data and scripts that are good starting points to use htrdr.

Release notes

Version 0.10

Use POSIX make as a build system

The build procedure is now written in POSIX make instead of CMake. In addition to the features already provided by its CMake alternative, the Makefile supports the use of static libraries and provides an uninstall target. In any case, the main motivation behind its writing is to use a good old well-established standard with simple features, available on all UNIX systems, thus simplifying its portability and support while being much lighter

Proof-reading and editing manual pages

Write the man pages directly in mdoc's roff macros, instead of using the scdoc markup language as a source for man pages.

Unlike writing manuals with man's roff macros, and even more so with scdoc, mdoc macros take care of layout, font handling and all the other typesetting details which, by construction, guarantee the consistency of all manuals without leaving the responsibility to the individual author. This also facilitates translation into other formats and documentation tools. These are the main reasons for writing manual pages with mdoc macros.

A complete re-reading of the manual pages was carried out during the translation into mdoc, with several corrections and rewrites to make the manual clearer.

Bug fixes

• Fix the construction of the planck/CIE cumulative: the memory space required could be prohibitive, leading to a shortage of memory space.
• Update error handling in htrdr_ran_wlen_planck_create to avoid freeing up the same memory space several times, and thus causing a crash.
• All dependencies have been updated. In particular, the htmi and rnatm libraries have been updated to version 0.1. They fixe several problems with the atmosphere and planeto commands. See their release note for more information.

Version 0.9.2

• Update the rnatm library to version 0.0.1. This versions fixes several bugs when different atmospheric components do not have the same volumetric meshes.
• Display an error message when parsing unknown arguments to the htrdr-planeto command.
• Fix htrdr-planeto man page: there was an error in the csplit command provided as an example to extract octrees from the output of htrdr-planeto when the -d option is used.

Version 0.9.1

• Fix invalid read/write memory access when ray tracing the ground in htrdr-atmopshere.
• Fix compilation warning detected by GCC 12.
• Fix htrdr-planeto man page (-S option): the unit of the radius and the distance from the source is not the meter but the kilometer.
• Fix htrdr man page: replaced long options with short options.
• Reference and install the rnrl fileformat man page.
• Proofreading the README and man pages: correcting typos, spelling and formatting errors and turns of phrase

Version 0.9

Adds radiative transfer simulation in 3D planetary atmospheres

The new htrdr-planeto command simulates radiative transfer in planetology context, i.e. in the 3D atmosphere of a telluric planet. Both infrared and visible computations are supported. htrdr-planeto is actually a renderer that calculates an image for a given observation position. Its internal rendering algorithm is based on Monte Carlo integration, which consists for each pixel in simulating a given number of optical paths from the sensor, taking into account the phenomena of light absorption and scattering.

The planet's ground can be any set of triangles with BRDFs and temperatures defined per triangle. The atmosphere is composed of a gas mixture and a potentially empty set of aerosols. Both can have arbitrary tetrahedral meshes with per-node radiative properties.

Miscellaneous

• Use scdoc rather than asciidoc as file format for man sources.
• Update all dependencies. More notably, use MruMtl 0.1 which introduces API breaks.
• Add the discrete wavelength distribution currently used in htrdr-planeto only.

Version 0.8.1

Sets the required version of Star-SampPling to 0.12. This version fixes compilation errors with gcc 11 but introduces API breaks.

Version 0.8

• Adds support for a thin lens camera model and an orthographic camera model for combustion and atmosphere modes.
• Updates the size of a tile from 32x32 pixels to 8x8 pixels. A tile is a block of pixels rendered by a thread. However, a size of 32x32 pixels could be too large when rendering on several dozen threads: the image definition could be insufficient to give tiles to all threads.
• Fixes the calculation of shortwave radiance by htrdr-combustion and the calculation of longwave radiance by htrdr-atmosphere. At each scattering position, the range of the traced ray could be incorrect.

Version 0.7

Adds the simulation of radiative transfer in combustion media

The new htrdr-combustion command performs radiative transfer computations in a scene representing a semi-transparent medium enlightened by a laser sheet. It uses Monte Carlo to calculate a monochromatic image of the medium or the radiative flux density. Both computations are performed in the visible at a given frequency.

The medium data are defined on the vertices of an unstructured tetrahedral mesh that may be surrounded by a triangular surface mesh representing the inner limits of the combustion chamber.

Updates the htrdr command

The previous htrdr command is renamed to htrdr-atmosphere. htrdr becomes a proxy for the htrdr-atmosphere command or the htrdr-combustion command: calling htrdr with the <atmosphere|combustion> options is equivalent to directly calling the htrdr-<atmosphere|combustion> commands.

Miscellaneous

• Major update of the entire codebase to add multiple applications to htrdr: It was originally designed to handle atmospheric applications only.
• Always displays the number of processes and the number of threads: previously they were only printed on multi-node executions.
• Fixed auto intersection issue on surfaces not facing the sun.
• Fixed writing of pixel data: assumed pixel layout could be wrong.

Version 0.6.1

• Fix the self-intersection issue in shortwave computations introduced by the 0.6 version.

Version 0.6

New features

• Add support of flux map computation for both shortwave and longwave. The flux is computed for the part of the flux map lying outside any geometry. The new command line option -p defines the rectangle in the scene onto which the flux is going to be integrated. The flux map resolution and the realisations count per pixel is controlled by the -i option.
• Add support of thin materials, i.e. materials without geometric thickness as for instance the leaves of the trees.
• Add the temperature property to the materials and used it as the limit condition during longwave computations. Previously, the surface temperatures were fetched from the atmosphere at the given surface position.
• Add the -n option to fix the name of the material defining the atmosphere.

Fix

• In shortwave, fix how direct contribution is handled for purely specular BRDF.

Version 0.5.1

• Fix the undefined strtok_r symbol issue exhibited by some GCC versions that leads to memory corruption and segmentation fault when parsing the ground interfaces.
• Fix typos in the man pages.

Version 0.5

New feature

Add support of shortwave integration with respect to the Planck function for a reference temperature whose default value is the blackbody temperature of the sun. Actually this is the counterpart of the longwave integration introduced by the "infrared rendering" in the 0.4 version. The main difference is that the source of radiation is the sun rather than the medium and its boundaries.

The option -l that enabled the infrared rendering is now replaced by the new -s option that controls the spectral integration that can be CIE XYZ (i.e. regular image rendering), longwave or shortwave.

Fixes

• Fix the returned sun radiance: the precomputed per spectral band solar incoming flux is removed and the sun radiance is now retrieved by directly evaluating the monochromatic Planck for the blackbody temperature of the sun.
• Fix CIE XYZ spectral integration: the pdf used to sample the CIE tristimulus values was not correctly handled in the Monte Carlo weight.
• Fix the longwave spectral integration: the Monte Carlo weight was wrong leading to overestimated temperatures.

Version 0.4

New features

• Add support of infrared rendering: when defined, the new -l option setups the range of longwave into which the rendering is performed. In infrared rendering, each pixel stores the radiance per pixel and its associated brightness temperature. Spectral integration is done with respect to the Planck function for a reference temperature of 290 K.
• The ground geometry can now have several materials whose data vary over the spectrum. These materials are listed in a new htrdr-materials file where each materials is defined by a name and a file storing its spectral data with respect to the MruMtl fileformat. A material is mapped to a part of the OBJ geometry by using the usemtl directive of the OBJ fileformat.
• Improve the sampling of the spectral dimension: the per wavelength realisation is now precisely sampled rather than arbitrarly fixed to the center of the sampled spectral band. Consequently, high resolution data defined per wavelength (e.g. Mie's properties and the reflectivity of the materials) are now fully taken into account.

Fixes

• Fix a deadlock when htrdr was run through MPI.
• Fix a memory leak: the output file was not closed on exit.

Version 0.3

• Add the -O option that defines the file where the sky data are cached. If the file does not exist, the sky data structures are built from scratch and serialized into this new file. If this file exists, these data structures are directly read from it, leading to a huge speed up of the htrdr pre-processing step. Note that if the provided file exists but is filled with data that do not match the submitted HTGOP, HTCP and HTMie files, an error is detected and the program stops.
• Rely on the HTSky library to manage the sky data. This library handles the code previously defined into the htrdr_sky.<c|h> files. The HTCP, HTGOP, HTMie libraries are thus no more dependencies of htrdr since only the htrdr_sky files used them.

Version 0.2

• Add the -b option that controls the BRDF of the ground geometry.
• Make optional the use of a ground geometry (option -g).
• Make optional the definition of the optical properties of water droplets (option -m) when no cloud field is used.

Version 0.1

• Add the -V option that fixes the maximum definition of the octrees used to partitioned the radiative properties of the clouds.
• Add a per pixel estimation of the per radiative path computation time.

Version 0.0.4

• Fix the computation of the surface scattering: there was a bug in how Russian roulette was implemented at surface scattering leading to an underestimation of the surface reflection.
• Update the thread allocation policy: by default, the number of threads is now defined as the maximum between the number of processors detected by OpenMP and the number of threads defined by the OMP_NUM_THREADS environment variable. This variable can be used to counteract the number of processors detected by OpenMP that can be lower than the real number of processors of the system.

Version 0.0.3

• Fix compilation on systems with a GNU C Library whose version is less than 2.19.
• Fix a possible invalid memory access to cloud data leading to segmentation faults.

History

htrdr has been funded by the ANR Rad-Net since 2021. |Méso|Star> is subcontractor of the project.

htrdr was funded by the ANR Astoria from 2018 to 2022. |Méso|Star> was sub-contractor of the project.

htrdr was funded by the ADEME (MODEVAL-URBA-2019) from 2019 to 2022. |Méso|Star> was partner of the project with CNRM.

htrdr was funded by the ANR High-Tune from 2016 to 2019. |Méso|Star> was sub-contractor of the project. Visit the High-Tune project web site.

Copyright notice

Copyright © 2018-2019, 2022-2023 Centre National de la Recherche Scientifique
Copyright © 2020-2022 Institut Mines Télécom Albi-Carmaux
Copyright © 2022-2023 Institut Pierre-Simon Laplace
Copyright © 2022-2023 Institut de Physique du Globe de Paris
Copyright © 2018-2023 |Méso|Star> (contact@meso-star.com)
Copyright © 2022-2023 Observatoire de Paris
Copyright © 2022-2023 Université de Reims Champagne-Ardenne
Copyright © 2022-2023 Université de Versaille Saint-Quentin
Copyright © 2018-2019, 2022-2023 Université Paul Sabatier

License

htrdr is free software released under the GPL v3+ license: GNU GPL version 3 or later. You are welcome to redistribute it under certain conditions; refer to the COPYING file for details.