## Playing with Jupyter and Darktable db… which focal lens do I like more?

I am trying to embed a full Jupyter notebook here… it should be more or less auto-explicative. Please comment if you need help!

Ok, so now: what new nice prime lens should I buy?

explore_data

cp \$HOME/.config/darktable/library.db dtlibrary.db

These are the available tables:

    sqlite> .tables
db_info          images           meta_data        used_tags
film_rolls       legacy_presets   selected_images

First of all, you load the database and save the tables you need in csv format. See for example http://www.sqlitetutorial.net/sqlite-tutorial/sqlite-export-csv/

    sqlite3 dtlibrary.db
SQLite version 3.22.0 2018-01-22 18:45:57
Enter ".help" for usage hints.
sqlite> .mode csv
sqlite> .output images.csv
sqlite> select * from images;
sqlite> .quit
In [1]:
# sources
# https://songhuiming.github.io/pages/2017/04/02/jupyter-and-pandas-display/
#
import sys
import os
# BEWARE only for command line interactive usage
from math import *
import scipy as sp
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
pd.set_option('display.height', 10)
pd.set_option('display.max_rows', 8)
pd.set_option('display.max_columns', 10)
pd.set_option('display.width', 80)

In [2]:
allphotos = pd.read_csv("images.csv")

In [3]:
allphotos

Out[3]:
id group_id film_id width height write_timestamp history_end altitude position aspect_ratio
0 8244 8244 295 5344 3516 1440610685 13 NaN 35407710388224 0.0
1 8245 8244 295 5344 3516 1440610554 15 NaN 35412005355520 0.0
2 8246 8244 295 5344 3516 1440610590 15 NaN 35416300322816 0.0
3 8513 8513 304 4928 3280 1451331582 10 NaN 36563056590848 0.0
5653 17653 17654 564 5184 3888 1565097337 0 NaN 75797582839808 NaN
5654 17654 17654 564 5240 3912 1565097337 0 NaN 75801877807104 NaN
5655 17655 17656 564 5184 3888 1565097337 0 NaN 75806172774400 NaN
5656 17656 17656 564 5240 3912 1565097337 0 NaN 75810467741696 NaN

5657 rows × 42 columns

In [4]:
allphotos.columns

Out[4]:
Index(['id', 'group_id', 'film_id', 'width', 'height', 'filename', 'maker',
'model', 'lens', 'exposure', 'aperture', 'iso', 'focal_length',
'focus_distance', 'datetime_taken', 'flags', 'output_width',
'output_height', 'crop', 'raw_parameters', 'raw_denoise_threshold',
'raw_auto_bright_threshold', 'raw_black', 'raw_maximum', 'caption',
'lightmap', 'longitude', 'latitude', 'color_matrix', 'colorspace',
'version', 'max_version', 'write_timestamp', 'history_end', 'altitude',
'position', 'aspect_ratio'],
dtype='object')

### Fix the database¶

A lot of entries for OLYMPUS are bad. I fix this here, but you can fix the database also with

check: select * from images where maker like "%OLYMPUS%" and crop=0;
change: update images set crop=2 where maker like "%OLYMPUS%" and crop=0;
In [5]:
allphotos.loc[(allphotos.maker=="OLYMPUS CORPORATION")&(allphotos.crop==0), 'crop']=2

In [6]:
allphotos['FLeq35'] = allphotos.crop * allphotos.focal_length

In [7]:
FLseries = allphotos[allphotos.FLeq35>0].FLeq35

In [8]:
FLs=np.array(FLseries)

In [9]:
FLs

Out[9]:
array([  33.        ,   33.        ,   31.99999899, ...,  250.        ,
140.        ,  140.        ])
In [10]:
plt.hist(FLs, bins =[0,20,40,60,80,120,200,500])

Out[10]:
(array([   21.,  3007.,   971.,   504.,   603.,   223.,   316.]),
array([  0,  20,  40,  60,  80, 120, 200, 500]),
<a list of 7 Patch objects>)

### Let’s try to separate them based on camera¶

In [11]:
cam_and_fl = allphotos[allphotos.FLeq35>0].loc[:, ['model', 'FLeq35']]

In [12]:
cam_and_fl

Out[12]:
model FLeq35
6 SLT-A55V 33.000000
7 SLT-A55V 33.000000
9 DMC-LX5 31.999999
10 SLT-A55V 45.000000
5653 E-M1MarkII 250.000000
5654 E-M1MarkII 250.000000
5655 E-M1MarkII 140.000000
5656 E-M1MarkII 140.000000

5645 rows × 2 columns

In [13]:
models = cam_and_fl.model.unique(); models

Out[13]:
array(['SLT-A55V', 'DMC-LX5', 'DMC-TZ100', 'TG-870', 'E-M5MarkII',
'E-M1MarkII', 'E-M10 Mark III'], dtype=object)
In [14]:
from matplotlib.ticker import ScalarFormatter
for i, mymod in enumerate(['SLT-A55V', 'DMC-LX5', 'DMC-TZ100',
'E-M1MarkII']):
fig, axs = plt.subplots(nrows=1, figsize=(5,3.5))
thisfls=cam_and_fl[cam_and_fl.model==mymod].FLeq35
L=np.array(thisfls)
axs.set_xscale("log")
axs.xaxis.set_major_formatter(ScalarFormatter())
axs.set_title(mymod)
axs.set_xticks([15, 24, 35, 50, 70, 100, 140, 200, 300])
axs.hist(L, bins =np.logspace(np.log10(10), np.log10(400), 20 ))


## Notes on upgrade from Ubuntu 14.04 to 16.04

I know that the standard policy in Ubuntu is to offer the upgrade from an old LTS to a new one when the new LTS reaches .1, i.e., 16.04.1, but well…

I upgraded my 14.04 in several machines, and I can signal these almost show-stopper problems (workaround exists for all of them, though):

on […]

## xkcd color list for LaTeX users…

After reading the (very) fun post about colors name at xkcd, I decided to create a small package to add the color list to my LaTeX documents…

After circumventing a small bug in xcolor, and ten lines of python, I have managed to create a small package that I am sharing here.

To use […]

## A new nice trick for the Linux keyboard: greek letters

I do not know you, but me (as an engineer) use Greek letters like α, β, γ and so on quite often. I did a small change to my linux keyboard layout: the new part in the keyboard symbols files now is

partial alphanumeric_keys xkb_symbols “standard_tlde” { include “es(basic)” key <TLDE> { [ grave, asciitilde, […]

## Eclipse

I suppose that everyone and their cat has eclipse shots this days… but I think this is a bit different:

I was shooting the March 20, 2015 eclipse (in Madrid) through a tree, and as a test I tried to focus on the tree silhouette instead — and that was the result. The normal, […]

## Managing PPAs in Ubuntu: a couple of useful scripts.

PPAs are really a great thing in Ubuntu: they allow you to have quite recent programs without upgrading all your system; most Ubuntu users use them extensively.

They have just one problematic side — the havoc that can happen on an upgrade. You see, when you upgrade your system, PPAs are disabled, but the program […]

## Quick fix xfig bug

Ok, so I am a computer dinosaur. I still use xfig for much of my drawings, although I recon it is a program from the old century…

Unfortunately, after the recent upgrade to Ubuntu 14.04, a nasty bug surfaced in xfig: a crash whenever you try to draw point-dashed lines.

## The perfect lens kit…

…does not exist, of course. But I think I have found mine; at least for my balance (as an amateur) among portability, flexibility and quality.

I am an amateur photographer; I do not sell photos (apart from some wet dream), and my needs are to produce photos for my family in the form of a […]

## How to modify a keyboard layout in Linux

Or: how to define a programmer-friendl(er) Spanish keyboard. …and you can add Greek, too!

A bit of history

There was a time when the computers all had the same keyboard — basically, the US one. You really do not need any more; the set of characters that could be represented where, after all, just the […]

## Shepherding at (very) small scale

I recently bought a (wireless) flashgun, a Nissin Di622 mark II for my Sony. And tried the Vivitar 100F3.5 Macro with a method I read from Robin Wong: I used the flashgun hand-held and the camera on the other hand. Firstly. Then I put the camera on a tripod, but with the flash still hand-held, […]