darktable article lede image

posts by Jo

local laplacian pyramids

improving contrast with the local laplacian filter

sometimes difficult lighting situations arise which, when taking photographs, result in unappealing pictures. for instance very uniform lighting on a cloudy day may give dull results, while very contrasty illumination (such as back lit) may require to compress the contrast to embrace both highlights and shadows in the limited dynamic range of the output device.

refer to the following two shots as examples:


compressing dynamic range with exposure fusion

modern sensor capture an astonishing dynamic range, namely some sony sensors or canon with magic lantern’s dual iso feature.

this is in a range where the image has to be processed carefully to display it in pleasing ways on a monitor, let alone the limited dynamic range of print media.

example images

use graduated density filter to brighten foreground

original

graduated density filter

using the graudated density iop works well in this case since the horizon here is more or less straight, so we can easily mask it out with a simple gradient in the graduated density module. now what if the objects can’t be masked out so easily?


colour manipulation with the colour checker lut module

[update 2016/07/31: there was a section about intermediate export to csv and manually changing that file. this is no longer needed, exporting the style directly from darktable-chart is fine now.]

motivation

for raw photography there exist great presets for nice colour rendition:

unfortunately these are eat-it-or-die canned styles or icc lut profiles. you have to apply them and be happy or tweak them with other tools. but can we extract meaning from these presets? can we have understandable and tweakable styles like these?


darktable 2.0 released

we’re proud to finally announce the new feature release of darktable, 2.0!

the github release is here: https://github.com/darktable-org/darktable/releases/tag/release-2.0.0

as always, please don’t use the autogenerated tarball provided by github, but only our tar.xz. the checksums are:

$ sha256sum darktable-2.0.0.tar.xz
d4f2f525bbbb1355bc3470e74cc158d79d7e236f3925928f67a88461f1df7cb1  darktable-2.0.0.tar.xz
$ sha256sum darktable-2.0.0.dmg
1019646522c3fde81ce0de905220a88b506c7cec37afe010af7d458980dd08bd  darktable-2.0.0.dmg

and the changelog as compared to the 1.6.x series can be found below.

when updating from the currently stable 1.6.x series, please bear in mind that your edits will be preserved during this process, but it will not be possible to downgrade from 2.0 to 1.6.x any more.


second release candidate for darktable 2.0

we’re proud to announce the second release candidate in the new feature release of darktable, 2.0~rc2.

as always, please don’t use the autogenerated tarball provided by github, but only our tar.xz.

the release notes and relevant downloads can be found attached to this git tag: https://github.com/darktable-org/darktable/releases/tag/release-2.0rc2 please only use our provided packages (“darktable-2.0.rc2.*” tar.xz and dmg) not the auto-created tarballs from github (“Source code”, zip and tar.gz). the latter are just git snapshots and will not work! here are the direct links to tar.xz and dmg: https://github.com/darktable-org/darktable/releases/download/release-2.0rc2/darktable-2.0.rc2.tar.xz https://github.com/darktable-org/darktable/releases/download/release-2.0rc2/darktable-2.0.rc2.dmg


released darktable 1.4.1

hi all,

as most of you probably noticed already, we published a point release, 1.4.1. the tarball is here:

https://sourceforge.net/projects/darktable/files/darktable/1.4/darktable-1.4.1.tar.xz/download

make sure you check the signature:

https://sourceforge.net/projects/darktable/files/darktable/1.4/darktable-1.4.1.tar.xz.asc/download

the macintosh computer disk image with signature here:

https://sourceforge.net/projects/darktable/files/darktable/1.4/darktable-1.4.1.dmg/download https://sourceforge.net/projects/darktable/files/darktable/1.4/darktable-1.4.1.dmg.asc/download

this is just a point release, so not really any new features. detailed release notes are as follows:

  • export: consistent names for output formats
  • export to disk: overwrite file option
  • grain plugin now allows smaller coarseness and will display coarseness values half of what they used to be, this is merely a cosmetic change, your images are unaffected.
  • some masks related fixes
  • some lua related fixes
  • tiff writer (32bit float, little endian output, configurable compression)
  • tiff reader
  • subtly nicer scrollbar behavior
  • theme loading cornercase fixups
  • shadow & highlight module improvements (should be less prone to artifacts when used on new images)
  • allow importing more than 1 style at a time
  • regression was fixed when building darktable against bleeding edge glibc
  • Sony A77V enhanced color matrix
  • Nikon D5100 updated white balance presets
  • Nikon 1 V2 noise profile (and by extension J3/AW1)
  • Nikon 1 J1 noise profile (and by extension V1/J2/S1)
  • Pentax K3 noise profile
  • experimental support for Panasonic DMC-LF1 (we still need samples for the nonstandard aspect ratios)
  • experimental support for SONY DSC-RX100M2
  • experimental support for SONY NEX-3N
  • still no Nikon D5300/D3300 support, we’re still looking into that.

the darktable release plus ppa is being phased out: https://launchpad.net/~pmjdebruijn/+archive/darktable-release-plus


determining focus in lighttable

wouldn’t it be great if you could judge sharpness of your images in lighttable mode? this mode is limited to small and medium sized thumbnails of your images, so we can deliver the required speed to browse a lot of them.

to tell whether or not you got the focus right during the shoot, we would like to look at the full resolution. the most you get out of lighttable mode will look like this:


about basecurves

the purpose of the basecurve is to make the otherwise scene-referred linear (linear raw rgb) color look good on your output devices. this is done independently of any color managed transforms which are also done in the pipeline, so we can establish a certain look independent of the devices. this will affect how highlights and shadows are balanced against each other, the overall contrast of the image, as well as color saturation. it basically boils down to:


have your lens calibrated!

just a quick plug for torsten’s great new service which allows you to calibrate your lens for lensfun. this will enable you to use darktable’s lens correction module with your lens if it hasn’t been calibrated by others for you yet.

all you need to do is take some sample images and upload them here:

http://wilson.bronger.org/calibration

the page contains some more detailed instructions about which images are useful and which aren’t.


released 1.2

we released the next feature release (1.2):

source tarball

user manual

macintosh disk image

as a feature release, it comes with a lot of new goodies:

  • profiled denoising: adapt to the properties of your camera’s sensor (72 cameras already profiled for you).
  • lightroom import: convert some basic edits from your lightroom collection to darktable operations.
  • multi instance support: duplicate your modules and apply them more than one time with different settings.
  • improved usability for distorting modules (streamline spot removal in the presence of crop/rotate for example).
  • selective copy/paste of image processing.
  • new more intuitive keystone correction tool.
  • jpeg2000 support.
  • graphics magick import (support virtually all input image formats).
  • much faster thumbnail loading (if you can live with crappy embedded thumbnails).
  • incredibly lengthy list of small bug fixes, performance enhancements, and usability improvements.
  • new camera support (decode and color matrices).
  • dithering against banding.
  • sharper thumbnails in lighttable mode.
  • new oauth2 based picasa uploader.
  • updated translations.
  • and a thoroughly overhauled user manual, proof read by natives (thanks heaps guys!).

this is the list of cameras supported for profiled denoising in this tarball:


released 1.2rc2

we just released a new tarball for the second release candidate:

https://sourceforge.net/projects/darktable/files/darktable/1.2/darktable-1.2~rc2.tar.xz/download

here is the macintosh disk image:

https://sourceforge.net/projects/darktable/files/darktable/1.2/darktable-1.2~rc2.dmg/download

and this time it comes with an updated usermanual:

https://sourceforge.net/projects/darktable/files/darktable/1.2/darktable-usermanual.pdf/download

  • the changes on top of rc1 are minor, some details:
  • fixed a spurious crash/deadlock when switching images in filmstrip mode
  • fixed a couple of exif strings for profiled denoising
  • fixed gcc 4.8.0 build
  • new noise profiles
  • fixed gphoto 2.5 issues
  • updated translations
  • updated user manual!

thanks all, and enjoy the release!


released 1.2rc1

we just released a new tarball for the first release candidate in the next feature release (1.2):

source tarball

mac disk image

as a feature release, it comes with a lot of new goodies:

  • profiled denoising: adapt to the properties of your camera’s sensor (72 cameras already profiled for you).
  • lightroom import: convert some basic edits from your lightroom collection to darktable operations.
  • multi instance support: duplicate your modules and apply them more than one time with different settings.
  • improved usability for distorting modules (streamline spot removal in the presence of crop/rotate for example).
  • selective copy/paste of image processing.
  • new more intuitive keystone correction tool.
  • jpeg2000 support.
  • graphics magick import (support virtually all input image formats).
  • much faster thumbnail loading (if you can live with crappy embedded thumbnails).
  • incredibly lengthy list of small bug fixes, performance enhancements, and usability improvements.
  • new camera support (decode and color matrices).
  • dithering against banding.
  • sharper thumbnails in lighttable mode.
  • new oauth2 based picasa uploader.
  • and the final release 1.2 will contain a thoroughly overhauled user manual, proof read by natives (thanks heaps guys!).

translations:


released 1.1.3

hi,

there is a new point release with a couple of smaller updates.

source tarball:

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1.3.tar.xz/download

mac disk image:

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1.3.2.dmg/download

and the usermanual is still the same.

fixes:

  • check (on build) if glib 2.28 or higher is present
  • don’t sanitize exif when creating hdr dngs
  • colorpicker now disappears immediately when disabling it
  • lens correction now uses loose lens matching (ivan tarozzi)
  • konica minolta dynax 5d rotation fix
  • removed an outdated assertion which could cause a crash in rare cases
  • don’t crash when loading half-corrupted xmps
  • don’t crash when an imported file contains incomplete gps information
  • libjpeg-turbo workaround (klaus post)

camera support:

  • preliminary support for the new nikon d5200

white balance presets:

  • sony alpha 700 (update to firmware v4)
  • sony alpha 230 (new)
  • canon eos 650d (new)
  • canon eos rebel t2i (fixed)
  • canon eos m (fixed)

enhanced color rendition:

  • konica minolta dynax 5d (wolfgang kuehnel)
  • sony nex 3 (wolfgang kuehnel)
  • sony alpha 230 (wolfgang kuehnel)
  • sony rx100 (josef wells)

darktable wouldn’t be where it is now if we weren’t able to depend on the great work of others, in particular we’d like to thank:


released 1.1.1

we released a patch release:

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1.1.tar.gz/download

along with an updated usermanual:

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-usermanual-1.1.1.pdf/download

this resolves a couple of issues with 1.1. so no new features here, but:

  • minor reordering of lighttable mode modules (geotagging, keywords and recent collections)
  • cleaned up the default visible plugins when first starting darktable
  • in most cases raw files will now show thumbnails in the import dialog (thanks to Mattias Eriksson)
  • a curve related crash was fixed (#9906 thanks to James C. McPherson)
  • comma seperated tags should work everywhere now (#9006 thanks to Tobias Ellinghaus)
  • Ulrich Pegelow fixed a huge amount of opencl related issues, particularly for AMD GPUs
  • we now deal better with hybrid GPU machines (#9074 by Ulrich Pegelow)
  • a deadlock in the lens correction module was fixed (#9106 thanks to Ulrich Pegelow)
  • we don’t delete module presets when cancelling the dialog anymore (#9108 thanks to Tobias Ellinghaus)
  • we now have better default memory usage settings (which are set upon starting darktable the first time)
  • initial support for SONY NEX 5R
  • preliminary/experimental Canon EOS 6D and Sony RX1 support (future changes for these camera’s may (for the time being) retroactively affect your images)
  • Canon EOS 6D white balance presets (thanks to no_maam_)
  • lots of updates for the usermanual (make sure you download a new copy from here)
  • and for our 1.1 the ubuntu packages from the PPAs were built without facebook export support, this has been fixed for 1.1.1

darktable wouldn’t be where it is now if we weren’t able to depend on the great work of others, in particular we’d like to thank:


profiling sensor and photon noise

… and how to get rid of it.

[update 02/05/2018 The information how to create camera noise profiles is outdated please read this tutorial instead!]

[update 20/12/2012: ‘how to profile your camera’ includes instructions with the new gen-profile script]

[update 15/12/2012: no more recompile needed, updated the instructions in the benchmark section and how to run make.sh.]

to summarize the current situation in dt: we have a lot of cool tools wrapped around great algorithms with almost all the knobs you need to get perfect results. while you can actually get really great results it’s this sheer number of knobs that makes finding a good parameter set quite a time consuming task. even creating per-iso presets is not straight forward, as most of the current modules depend on a lot more stuff early on in the pipe (whitebalance, exposure, basecurve, etc).


released darktable 1.1

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1.tar.gz/download

this is a feature release, so there is a lot of new stuff:

general

  • new camera support, new whitebalance presets, etc., including canon eos m support and samsung nx fix
  • similarity matching search for images that look alike.
  • geotagging, complete with map view (thanks to dinamic for starting that ages ago and to houz for actually bringing it home): “ Geotagging in darktable
  • mac os package: “ Bringing current darktable to OS X
  • a lot of bugfixes (mainly thanks to ulrich for his meticulous work)
  • facebook exporter (for those who have an account there)

ui

darkroom

  • extensive use of edge-aware filtering techniques to suppress noise, halos and ringing all around darktable: “ edge aware image development
  • conditional blending, and a lot of goodies around it! “ Some enhancements to conditional blending
  • magenta highlights: “ magenta highlights ” improved on high-contrast edges to overexposed areas (should get rid of purple highlights on tiny water waves and purple fringes around tree leaves for example)
  • much improved sharpness for both export and darkroom view, especially for downsampled images and if you use lens corrections or rotations/perspective corrections. check the new options in the preferences dialog, also one more than mentioned in the blog (“demosaicing for zoomed out darkroom mode” to trade performance for even more sharpness): “ Upcoming features: New interpolation modes and better resize

color management

  • improved per-screen color management (should reload the screen profile automatically)
  • more compatible embedded color profiles (should fix problems on windows viewing our images, if that matters)
  • read embedded color profiles from jpg

opencl

  • most of our modules now can take advantage of your computer’s gpu power
  • caching for compiled opencl kernels (even in case the driver doesn’t do it) for faster startup times

usermanual

translations

  • two new translations (both portuguese … ;) )
  • well translated: cs de es fr it ja nl pl pt_BR pt_PT sv
  • half translated: ca fi gl ro ru sq th zh_CN

enjoy!


released 1.1rc2

i just uploaded the tarball for the second release candidate for 1.1:

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1~rc2.tar.gz/download

packages for your favourite distros should be in the usual place, mac dmg is here:

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1~rc2.dmg/download

i know there are still translations coming, but something has to be left for the final release after all :)

changes since rc1 aren’t many as to be expected, but we’ve got:

  • canon eos m support (new rawspeed, also includes samsung nx fix)
  • usermanual is progressing
  • two new translations (both portuguese … ;) )
  • reworked the much hated `more plugins’ widget (thanks to boucman)
  • a lot of bugfixes (mainly thanks to ulrich for his meticulous work)

enjoy!


1.1 release candidate 1

as commits are easing down a little lately it seems appropriate to push out the first release candidate of the new version:

https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1~rc1.tar.gz/download

[update: mac package is available from https://sourceforge.net/projects/darktable/files/darktable/1.1/darktable-1.1~rc1.dmg/download ]

That doesn’t mean this tarball is final or perfect by any means. we’ll probably go on releasing a couple of these on our way to the final release in a few weeks. This will be a feature release, as opposed to the bug fixing point release series 1.0.x, so we’ve got a huge bunch of changes for you this time. The changelog is not the final one either, but here it is:


edge aware image development

in an ideal world, an image is piecewise smooth. it has soft gradients, some detail and edges. in particular there’s no noise and the edges are sharp. given these assumptions, you can do a lot of cool things to your pictures, using techniques like frequency space editing, wavelets, or local histograms.

darktable’s equalizer module demonstrates some of this, using the wavelet approach. you can use it to sharpen and denoise, enhance or attenuate certain frequencies in your image, while keeping the edges intact.


darktable 1.0.5 released

It’s our pleasure to announce that darktable-1.0.5 has been released. Find the tarball on sf.net:

https://sourceforge.net/projects/darktable/files/darktable/1.0/darktable-1.0.5.tar.gz/download

The Ubuntu PPAs have been built already, you should get them with your next update automatically if you subscribed to Pascal’s PPA.

This has a good chance of being the last one in a series of stable releases (with stuff backported from our latest and greatest). Thanks to Pascal for maintaining it! As such, it comes with a short list of maintenance things as change log:


magenta highlights

false color highlights seem to be an issue frequently, so here’s some quick faq about it. alexandre, please excuse all the outward references ;)

why are my highlights magenta?

that’s how the sensor works. it collects a couple of photons, at some point it fills up and rejects to deliver any more useful information past this point. unfortunately that doesn’t happen at the same time for all color channels.

how does the sensor work?

usually digital cameras come with a color filter array (CFA) of absorptive filters in front of an array of CCD or CMOS sensor which collect electrons proportional to the incoming photons according to the photoelectric effect.


color correction

this is one of the oldest modules in darktable. it appeared to me that it probably lacks an example to discover how useful it can be … so here goes the example.

this started off to be a wrapper around the gegl:whitebalance operation, which works in Lab color space and is able to give dark and bright colors a different color tint, interpolating between the two for mid tones.

so suppose you have the following image:


bauhaus widgets

disclaimer: this is only to tease you and will not make it into the next release, but the one after …

when reading gui-guidelines, most of them seem to be too general, or too specific for a certain kind of programming environment (gnome and gtk, qt, etc).

for our purposes, i found the fundamental principles of the bauhaus school to be more appropriate. radical simplicity, no unnecessary shape or line, such as a pseudo 3d-bevel-border around ui elements. the underlying grid should be visible because all elements are aligned, not because it is drawn. only simple shapes are allowed, and everything should integrate seamlessly into the background of the panel.


why you want raw

or: how to rescue your shot after the fact.

also: how to use color zones for black and white.

sometimes i’m surprised by what kind of data is hidden in my raw images, and i want to pass this on to those of our users who happily take pictures in jpg. actually it’s just a short story about a typical communication problem between me and my camera and the way darktable moderates that, after the fact.


Released 0.9.3

As most of you probably noticed by now, we released 0.9.3. The tar file can be found here: https://sourceforge.net/projects/darktable/files/darktable/0.9/darktable-0.9.3.tar.gz/download

Pascal updated his ppa for ubuntu here: https://launchpad.net/~pmjdebruijn/+archive/darktable-release or https://launchpad.net/~pmjdebruijn/+archive/darktable-release-plus (with Exiv2 0.22, and Lensfun 0.2.5 + lens data from svn).

and made great new screencasts explaining a couple of features and differences to 0.9.2. You can find them on our resources page or Pascal’s website

It is a comparatively minor update to our stable 0.9.x series. It does not contain many very cool new features and big changes we have in git, because we don’t consider them stable enough yet. Nontheless, it contains 272 commits over the previous release 0.9.2, mainly containing:


darktable and research

you might have noticed our equalizer tool, and been confused by it and the many controls. that’s probably partly because you didn’t see a similar thing before, we had to develop it first.

very short history

behind the ui is a powerful frequency domain processing technique, based on wavelets. the most commonly used wavelets are based on the lifting scheme [swe97], work in a data-independent way, and are decimated (i.e. the coarse coefficients are much more sparse than the fine ones). while this allows for very fast implementations, data-independent wavelets can lead to blurring or ringing around edges (depending on what you do to the coefficients during image enhancement). raanan fattal had quite an inspiring paper at siggraph 2009 [fat09] introducing edge weights into the lifting scheme to overcome this. while the method is fast, produces okay results (the legacy equalizer version I was based on this), it has some problems caused by the decimation. in particular, decimated wavelets are not shift-invariant, which means your results will change if you slightly crop the image for example. actually the same author also had a solution for that earlier already [far07], in a different context, and not quite as fast.


a new caching backend

since i probably tend to make this more technical than any reader would like to, here’s the take home message:

  • much faster import of folders
  • much faster thumbnail creation for first-time images
  • much improved scalability wrt concurrency
  • much improved scalability wrt total number of images in your database (should be good up to around 500 million images)
  • much improved robustness (no more deleting ~/.cache/darktable/mipmaps all the time, yay)

some context:

darktable’s light table mode shows you your image collection in arbitrary order and filtered by arbitrary queries to the underlying database. that means that there is quite some uncertainty about which thumbnails you are going to need in the next second. so we rely on a caching mechanism that stores a few thumbnails, regenerates them as needed, and evicts not so frequently used thumbnails from the cache.