darktable 3.6.0 released

We’re proud to announce the new feature release of darktable, 3.6.0!

The github release is here: https://github.com/darktable-org/darktable/releases/tag/release-3.6.0.

As always, please don’t use the autogenerated tarball provided by github, but only our tar.xz file. the checksums are:

$ sha256sum darktable-3.6.0.tar.xz
86bcd0184af38b93c3688dffd3d5c19cc65f268ecf9358d649fa11fe26c70a39 darktable-3.6.0.tar.xz
$ sha256sum darktable-3.6.0.2.dmg
51fbde31f1a5551774a2b7ed8693ab906d7d37d509dddd1c2bfd2b8746cdd4a6 darktable-3.6.0.2.dmg
$ sha256sum darktable-3.6.0-win64.exe
aad925988545ff3c50f54243520e8481c695b38b209daa808a490113f6400374  darktable-3.6.0-win64.exe

When updating from the currently stable 3.4.x series, please bear in mind that your edits will be preserved during this process, but the new library and configuration will not be usable with 3.4.x any more.

You are strongly advised to take a backup first.

Important note: to make sure that darktable can keep on supporting the raw file format for your camera, please read this post on how/what raw samples you can contribute to ensure that we have the full raw sample set for your camera under CC0 license!

• Almost 2680 commits to darktable+rawspeed since 3.4
• 954 pull requests handled
• 290 issues closed

The Big Ones

• Full rework of the import module. The new module allows you to see thumbnails of your images before import and makes it easier to handle multiple import tasks. The same dialog is now used for importing from disk, card or camera.

  When importing “in-place”, images that are already present in darktable are flagged and can be automatically excluded from the import.

  Importing from a local drive won’t use gphoto and so should work more reliably on Windows.

• A vector-scope is added to complement the current histogram, waveform and parade views.

• A new default demosaic algorithm (RCD) has been introduced, which should provide better quality than the current default (PPG) without the long run-times of AMaZE. the order of the demosaic algorithms has also been adjusted for clarity.

• Support for “dual demosaic” has also been added, allowing you to combine “RCD + VNG4” and “AMaZE + VNG4” for bayer sensors, for xtrans sensors there is “Markesteijn 3-pass + VNG”. This allows you to use an algorithm better resolving fine detail and another one delivering smoother output depending on local image content.

• You can now create an opacity mask based on the sharpness of the image. This uses a similar algorithm to that used in dual demosaic.

• Drawn shapes are drawn on the original RAW image and then distorted through the pixelpipe. This previously led to, for example, circles becoming ellipses (due to lens distortion) after being placed on the image. All drawn shapes can now be pre-visualized and moved prior to placement, with full deformation applied. This makes the editing of masks easier fully WYSIWYG.

• A new Color Balance RGB module has been added. This can be used as a one stop module for all color grading work.

• A new censorize module has been added. This allows you to hide parts of the image by adding a blur/pixelization.

• A new crop module has been added. This new module has been added late in the pixelpipe and makes it possible to use cropped parts of the image for source spots in the retouch module, for example.

• A new section has been added to the color calibration module to allow you to calibrate colors using a color checker chart.

• The map module has been enhanced in a number of ways:

  • When importing a GPX the trace can be displayed.
  • Location can be set using polygons and directly created from information reported by Open Street Map. As a reminder, all locations create tags automatically for images placed into the area.

• The basic adjustments module has been replaced with the quick access panel, which can be used to group controls from multiple different modules in one place. Currently, only non-graphical controls (sliders, comboboxes etc.) can be used.

Other New Features And Changes

• The various lighttable modes can now be accessed via clear and descriptive icons at the bottom of the screen. These should be easier to use than the existing drop-down selection.

• The export module is now also accessible on the left-hand panel of the darkroom view.

• The mix control in the contrast equalizer module is now retained as module parameter. This means that no data is lost when using this slider and you can always change the mix later with full precision.

• The following modules have been deprecated:

  • The basic adjustments module has been replaced by the new quick access panel.

  • The vibrance module has been replaced by the vibrance control in the new color balance rgb module.

  • The spot removal module has been replaced by the retouch module (use this module in “clone” mode to replicate the spot removal functionality)

  • The defringe module has been replaced by a new “chromatic aberrations” module. The old module has been renamed to “raw chromatic aberrations”.

• The RAW chromatic aberration module has been enhanced to support multiple iterations and an option to help avoid color shifting has been added (not activated by default).

• Scene-referred workflow is now the default, though this can still be changed in preferences.

• The ESC key can be used to exit the slideshow view.

• You can now Ctrl-Click on the module groups preset button to access the manage dialog directly.

• Many aspects of the GUI have been reworked or tweaked for a better user experience (combo-box alignment, padding, histogram icons, etc.).

• The color of scope views (histogram, parade, waveform) has been reworked using CSS.

• The integrated help system now points to the new user manual which is up-to-date for this release. Also for development builds of darktable these links will go to the development version of the manual. So more links have been added, for example, for the lighttable layout and the timeline.

  As a reminder, the new documentation is available in the following location (viewable online or in pdf and epub formats):

  https://www.darktable.org/usermanual/3.6/en/

• Many graphs in processing modules can now be resized with Ctrl+Scroll. (Filmic RGB, RGB levels, levels, lowlight, colorzones, rawdenoise, denoise profile, contrast equalizer).

• A new color preservation method (“norm-preserving colorfulness”) has been added to the color calibration module.

• A simplified and more precise message has been added to notify the user when their database(s) are locked by another process.

• A rational function has been added to toe/shoulder controls in Filmic RGB.

• A new preference for advanced users can now be used to disable the module warning messages. Use at your own risk.

• Preliminary work toward full support for CR3 has begun. darktable’s exif support can now read CR3 files if the proper exiv2 library version is found.

• A new color science with norm preservation under de-saturation has been added to Filmic RGB and a new scaled euclidean norm is supported.

• Some changes have been made to make liquify faster when editing nodes. Also smaller deformation stamps are now supported.

• The style of module sub-sections has been tweaked so that they stand out more.

• It is now possible to more easily manage overlapping masks. For example, it is possible to change an ellipse’s control points when they lie within a larger circle or path shape.

• Easier handling of the mask controls - do not require high precision as mouse collision detection has been reworked.

• The preferences, “manage module groups” and import dialogs will remember and restore their size after closing. Their default size is designed to support smaller screens and it is recommended that you resize them as appropriate.

• A huge number of internal optimizations have been made throughout the code. For example: soften, tone equalizer, drawn masks, color picker, color space conversions. It is impossible to list them all

• Many speed improvements have been made for the standard CPU code path by tuning the OpenMP code. There are some huge gains with the new code in some modules. We have then removed some hand-coded SSE routines as they are not faster and are hard to maintain.

• The prompt that appears when exporting in overwrite mode is now optional and controlled by a security preference.

• An icon has now been added to module headers to indicate whether a module uses a mask. This allows you to quickly see which modules use a mask and to enable/disable the mask preview by clicking on it.

• A message is now shown when a camera is plugged and darktable is not able to connect to it. The general cause is that the camera is mounted by the OS and so is exclusively locked.

• The grey darkroom loading screen is now optional. This can lead to on-screen artifacts but can also be useful to quickly compare versions of an image.

• Module group presets can now be auto-applied based on the image being edited. This make it possible to create groups specifically to develop color, monochrome, RAW or Jpeg images, for example.

• Different interpolation algorithms are advised for warping and scaling modules. A new preference has been added to allow you to select these algorithms independently.

• RGB scene blending mode is now used for the denoise (profiled) presets.

• Parametric masks are now handled better when being reset. Some parametric masks may need to be inverted in order to be considered as a no-operation.

• The GUI of the blending section has been rewored to be more compact.

• Distortion is no longer computed for shapes that are outside of the current view.

• Better values are reported on toast messages when changing masks’ size and feather.

• Undo has been enhanced to support duplicates. That is, the creation of duplicates can be undone using Ctrl+Z (undo).

• In the darkroom, undo now restores the mask display status along with the module changes.

• Non-matrix-based color profiles have been filtered out of the histogram profile menu.

• The timeline/filmstrip keyboard shortcut has been changed to Ctrl-B. This is more consistent with Ctrl+Shift+B (used to toggle the bottom panel) and avoids a conflict with sticky preview shortcut.

• Some preferences have been moved to a sub-menu in the module preset menu. This makes the global preferences lighter. This has been done for the import, export, metadata and “collect images” modules.

• Jpeg-2000 files are now decoded faster (using multiple threads).

• The lighttable has been enhanced to use the embedded Jpeg preview depending on the size of the thumbnails being displayed. Setting such a preference to use embedded preview for small thumbnails may speed-up the handling of large collections during import, for example.

• Added support for removing multiple styles at once.

• Added support for import/export of presets on mass.

• Added collect filter presets to display images based on aspect ratio and import date.

• The variables $(EXIF_xxx) can now be used while importing pictures.

• Graduated density module’s control is fixed when using down-sampling.

• Color zone saturation has been adjusted to avoid misleading visual illusions.

• Modules can now be removed from a group by right-clicking on the group icon.

• Many improvements have been made to the TIFF export module.

• In the lighttable, when several images are selected, the image information module now displays the information only if every image share the same data. If data are different, a <various values> text is displayed.

• In the export module, the last visited Piwigo album is now remembered between sessions.

Bug Fixes

• Fix ellipse and gradient rotation angle when distortion modules are activated. This is a GUI issue only when adding and editing shapes.

• Fix white-balance from old edits where the white-balance module was left at default values in darktable 2.6.x. For those edits we need to recover the legacy values and not the new ones from 3.x.

• Use smooth zooming to avoid thumbnails flickering.

• Fix entering of exponent expression in sliders for keyboard where ^ is a dead key (this is the case in the German and French layout for example).

• The import should be a bit faster for large collections by limiting the GUI update.

• Fix various AVIF issues (set YUV range, proper use of color primaries, set codecName, etc.)

• Fix demosaic border handling and have fewer differences between the CPU & GPU code path.

• Fix use of sources in spot and retouch when distortion is used. The source area was not always correctly distorted and was actually giving different results depending on whether distorting modules were present in the pipe.

• In the default watermark simple-text we now preserve the spaces.

• Properly check for CLUT in all ICC intents before extracting matrix.

• Properly use the inverse-transform routine provided by lensfun instead of an adhoc iterative routine in the lens correction module. This makes the code cleaner and probably more precise.

• Hide borders by default in slideshow view.

• Ensure that duplicates are given the same time-stamp as the source image.

• Fix creation of duplicates when the workflow default has been changed since the base image was created. That is, an image created with the display-referred workflow must be fully identical when duplicated, even if the workflow default has been changed to scene-referred.

• Properly group focal lengths in the collect images filters.

• Better naming of the RAW prepare module controls.

• Fix live-view display scaling on large screens (especially for HiDPI ones).

• Fix migration of legacy parameters for denoise (profiled). A copy/paste error was copying the shadow parameter as the new bias.

• Fix possible loss of masks where a module is disabled just before switching to another module.

• A new module instance can now be created by right-clicking on the multi-instance menu or on a preset. This replaces middle-click, which is not available on all mice.

• Fix zoom in the lighttable full preview to keep the mouse position.

• Recover forms from legacy spot in version 1.

• Fix reset in the export module so that it resets all module parameters to their default values.

• Fix HDR DNG white balance setting.

• Import Lightroom metadata only if there is not already a darktable XMP for the corresponding image.

• Fix color profile handling in the overexposed module.

• Fix style undo in overwrite mode.

• Fix possible issue with CUPS printer margins computation when computer is not using the English locale.

Notes

• The database upgrade can be slow.

  This new version modifies quite a lot the database structure by adding foreign keys and cascade support. This will avoid dangling data in the database and will enforce a better consistency.

Lua

• API changed to 6.2.0

• Password storage and retrieval - added darktable.password.save() and darktable.password.get() to save and retrieve passwords using the darktable password storage back end.

• Widget naming - widgets now have a name field, making it possible to style them using CSS

• darktable.gui.libs.visibility is fixed so that setting a lib to hidden (false) removes it from the user interface.

• Preferences - added darktable.preferences.destroy() to remove a preference. Added darktable.preferences.get_keys() to return a sorted table of all of the preferences for both darktable and lua.

• Get image from database by image id - added darktable.database.get_image() to retrieve an image by the image ID

• darktable.gui.libs.metadata_view.destroy_info() - function added to remove field added by darktable.gui.libs.metadata_view.register_info() from the image information display. Thanks @phweyland

• API change to 6.2.1 - SCRIPT BREAKING CHANGE

• Event naming - add a name field to darktable.register_event() so that multiple events of the same type can differentiated and the desired one selected and accessed.

• Event destruction - added darktable.destroy_event() to remove an event.

• API change to 6.2.2 - SCRIPT BREAKING CHANGE

• Add selection-changed to the lua events so that scripts can respond when the selection changes

• Selection naming - add a name field to darktable.gui.libs.select.register_selection() so that the selection can be selected and manipulated after creation

• Selection destruction - add darktable.gui.libs.select.destroy_selection() to remove an selection button from the user interface.

• Selection sensitivity - add darktable.gui.libs.select.set_selection_sensitive() to set selection button sensitivity

• API change to 6.2.3 - SCRIPT BREAKING CHANGE

• Action naming - add a name field to darktable.gui.libs.image.register_action() so that the action can be selected and manipulated after creation

• Action destruction - add darktable.gui.libs.image.destroy_action() to remove a lua created action button from the user interface.

• Action sensitivity - add darktable.gui.libs.image.set_action_sensitive() to set action button sensitivity

• Bugfix - Changed text_view widget to wrap lines when the text exceeds the width of the widget

• Storage removal - add darktable.destroy_storage() to remove a storage from the exporter

• Widget visibility - add visible member to widgets for visibility control

• Change API to 7.0.0 for darktable 3.6.0 due to all of the breaking changes this development cycle

Changed Dependencies

RawSpeed changes

Camera support, compared to 3.4.1

Base Support

• Fujifilm GFX100S (compressed)
• Fujifilm X-E4 (compressed)
• Fujifilm X-S10 (compressed)
• Leica D-Lux 7 (4:3, 1:1, 16:9, 3:2)
• Nikon Z 6_2 (14bit-compressed, 14bit-uncompressed, 12bit-compressed, 12bit-uncompressed)
• Nikon Z 7_2 (14bit-compressed, 14bit-uncompressed, 12bit-compressed, 12bit-uncompressed)
• Panasonic DC-G100 (4:3)
• Panasonic DC-G110 (4:3)
• Panasonic DC-GF10 (4:3)
• Panasonic GF90 (4:3)
• Pentax K-3 Mark III
• Ricoh GR III (dng)
• Sony ILCE-1
• Sony ILCE-7SM3
• Sony ILCE-QX1

White Balance Presets

• Fujifilm GFX100S
• Fujifilm X-E4
• Nikon D780
• Panasonic DC-FZ10002
• Panasonic DMC-G8
• Panasonic DMC-G80
• Panasonic DMC-G81
• Panasonic DMC-G85
• Ricoh GR III
• Samsung GX20

Noise Profiles

• Canon PowerShot S110
• Nikon COOLPIX P950
• Nikon Z 7_2
• Olympus E-M10 Mark IV
• Pentax K-3 Mark III
• Samsung GX10
• Samsung GX20
• Sony ILCE-1
• Sony ILCE-7C

Custom Color matrices

• Samsung GX20

Translations

• Afrikaans
• German
• Esperanto
• European Spanish
• French
• Hebrew
• Hungarian
• Italian
• Dutch
• Brazilian Portuguese
• Russian
• Slovenian
• Ukrainian