lensserious_db.h File Reference

A read-only database, and an API with nothing behind it. More...

#include "lensserious.h"

Include dependency graph for lensserious_db.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Data Structures
struct	ls_camera_t
struct	ls_db_match_t

Typedefs
typedef struct ls_db_t	ls_db_t
typedef struct ls_camera_t	ls_camera_t
typedef struct ls_db_match_t	ls_db_match_t

Functions
size_t	ls_db_normalize (const char *in, char *out, size_t out_size)
	The comparison form of a name: ASCII case-folded, punctuation dropped, runs of whitespace collapsed to one space.
int	ls_db_tokenize (const char *norm, char *out_tokens, int max, int stride)
	The tokens ls_db_match_lens() compares, from an already-normalised name.
unsigned	ls_db_token_hash (const char *token)
	FNV-1a of a token, the hash the stored digest and the matcher both use.
size_t	ls_db_token_digest (const char *norm, unsigned char *out, size_t out_size)
	Pack a normalised name's tokens into the digest stored in lens_name.tokens.
ls_db_t *	ls_db_open (const char *path)
	Open a database for reading.
void	ls_db_close (ls_db_t *db)
	Release a handle. Safe on NULL.
const char *	ls_db_error (const ls_db_t *db)
	The last error on db, as a string owned by db, or NULL.
int	ls_db_schema_version (const ls_db_t *db)
	Schema version of the open file, or -1. Bumped when the layout changes.
int	ls_db_meta (ls_db_t *db, const char *key, char *out, size_t out_size)
	A meta value by key (built_utc, source, lensfun_db_version, ...).
int	ls_db_find_camera (ls_db_t *db, const char *maker, const char *model, ls_camera_t *out)
	Find a camera by maker and model.
int	ls_db_find_lens (ls_db_t *db, const char *maker, const char *model, float crop, ls_lens_t *out)
	Find a lens by maker and model, and fill out with its coefficients.
int	ls_db_match_lens (ls_db_t *db, const char *maker, const char *model, long long mount_id, float crop, ls_db_match_t *out, int max)
	Find the lenses a free-text name most likely refers to.
int	ls_db_lens_by_id (ls_db_t *db, long long lens_id, ls_lens_t *out)
	Load a lens by its database id, for a caller that already resolved one.
int	ls_db_lens_fits_mount (ls_db_t *db, long long lens_id, long long mount_id)
	Does a lens fit a camera's mount, upstream's compatibility table included?
int	ls_db_list_lenses (ls_db_t *db, long long *out_ids, int max)
	Enumerate lens ids, oldest-inserted first, for tests and for a GUI's lens picker.
int	ls_db_lens_name (ls_db_t *db, long long lens_id, char *maker, size_t maker_size, char *model, size_t model_size)
	The lens's maker/model, as stored.
int	ls_db_lens_range (ls_db_t *db, long long lens_id, float *min_focal, float *max_focal, float *min_aperture, float *max_aperture)
	The lens's focal and aperture range, for a picker that lists it.
int	ls_db_lens_mounts (ls_db_t *db, long long lens_id, char *out, size_t out_size)
	The mounts a lens is made for, joined with ", ".
int	ls_db_list_cameras (ls_db_t *db, long long *out_ids, int max)
	Enumerate camera ids, for a GUI's camera picker.
int	ls_db_camera_name (ls_db_t *db, long long camera_id, char *maker, size_t maker_size, char *model, size_t model_size, char *variant, size_t variant_size)
	A camera's maker, model and variant, as stored.
int	ls_db_camera_by_id (ls_db_t *db, long long camera_id, ls_camera_t *out)
	Load a camera by its database id, for a caller that already resolved one.
int	ls_db_mount_name (ls_db_t *db, long long mount_id, char *out, size_t out_size)
	A mount's name.
int	ls_db_lenses_for_mount (ls_db_t *db, long long mount_id, long long *out_ids, int max)
	The lenses made for one mount.

Detailed Description

A read-only database, and an API with nothing behind it.

What this deliberately is not: lensfun's lfDatabase, a process-wide object that must be constructed before anything can be asked, that parses ~8 MB of XML into 1051 cameras and 1562 lenses on the way up (measured 89-102 ms and +4 MB resident), and that every caller must then serialise against. In Ansel that object became one global plus one mutex held across every lookup, which is precisely the shape this library exists to remove.

Here the database is a file, and this is how you read it:

• Stateless. No initialisation, no shutdown, no global. Every function is a pure function of the file and its arguments, and every result is a plain value the caller owns outright. Nothing is cached between calls, so nothing can go stale.
• Read-only, and lock-free. The file is opened mode=ro&immutable=1, which tells SQLite the bytes cannot change underneath it: no WAL, no shared-memory segment, no file locks, no rollback journal. Combined with SQLITE_OPEN_NOMUTEX there is no mutex anywhere on the read path – not one this library takes, and not one SQLite takes for it.
• Any path. A filesystem path or a full file: URI, wherever the caller keeps it. A process can read several databases at once with no interaction between them.

The one rule

immutable=1 is a promise about the FILE, and SQLITE_OPEN_NOMUTEX is a promise about the HANDLE. So:

• One handle per thread. A handle is cheap (ls_db_open does no parsing; it opens a file). Threads must not share one – there is no mutex to make that safe, which is the point. Handles to the same file from different threads are fine and do not interact.
• A database file is replaced, never edited. The rebuild writes a new file and renames it into place, so an open handle keeps reading the inode it opened and a handle opened during the rename gets one whole version or the other. Editing a live database in place breaks the immutable promise and the reader will return wrong answers rather than fail – do not do it. tools/import_lensfun_xml.c does the write-then-rename itself.

Definition in file lensserious_db.h.

Typedef Documentation

ls_camera_t

typedef struct ls_camera_t ls_camera_t

A camera, as far as a correction is concerned.

ls_db_match_t

typedef struct ls_db_match_t ls_db_match_t

One candidate from ls_db_match_lens(), best score first.

ls_db_t

typedef struct ls_db_t ls_db_t

An open database. Not shared between threads; see The one rule.

Definition at line 103 of file lensserious_db.h.

Function Documentation

ls_db_camera_by_id()

int ls_db_camera_by_id	(	ls_db_t *	db,
		long long	camera_id,
		ls_camera_t *	out )

Load a camera by its database id, for a caller that already resolved one.

Definition at line 616 of file lensserious_db.c.

References _db_err(), ls_camera_t::crop_factor, ls_camera_t::id, ls_camera_t::mount_id, and ls_db_t::sql.

ls_db_camera_name()

int ls_db_camera_name	(	ls_db_t *	db,
		long long	camera_id,
		char *	maker,
		size_t	maker_size,
		char *	model,
		size_t	model_size,
		char *	variant,
		size_t	variant_size )

A camera's maker, model and variant, as stored.

Parameters

db	an open database.
camera_id	the camera.
maker,model,variant	caller's buffers; any may be NULL. variant is set to the empty string for the cameras that have none, which is most of them.
maker_size,model_size,variant_size	their sizes.

Returns

1 when the camera exists, 0 when it does not, -1 on error.

Definition at line 585 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_close()

void ls_db_close

(

ls_db_t *

db

)

Release a handle. Safe on NULL.

Definition at line 126 of file lensserious_db.c.

References ls_db_t::sql.

Referenced by ls_db_open().

ls_db_error()

const char * ls_db_error

(

const ls_db_t *

db

)

The last error on db, as a string owned by db, or NULL.

Definition at line 133 of file lensserious_db.c.

References ls_db_t::error.

ls_db_find_camera()

int ls_db_find_camera	(	ls_db_t *	db,
		const char *	maker,
		const char *	model,
		ls_camera_t *	out )

Find a camera by maker and model.

Matching is exact on the normalised form of each name – case-folded, with punctuation and runs of whitespace collapsed – against every spelling the database holds for that camera, translations included. maker may be NULL to search on the model alone.

This is NOT lensfun's fuzzy scorer, which also tolerates missing words, reordered tokens and vendor prefixes. Porting it is separate work; until then a caller that needs that behaviour should treat a miss here as "not found" rather than as "no such camera".

Returns

1 on a match, 0 if nothing matched, -1 on error.

Definition at line 374 of file lensserious_db.c.

References _db_err(), ls_camera_t::crop_factor, ls_camera_t::id, ls_db_normalize(), ls_camera_t::mount_id, and ls_db_t::sql.

ls_db_find_lens()

int ls_db_find_lens	(	ls_db_t *	db,
		const char *	maker,
		const char *	model,
		float	crop,
		ls_lens_t *	out )

Find a lens by maker and model, and fill out with its coefficients.

Same matching as ls_db_find_camera(). When several lenses share a name – upstream distinguishes them by the sensor they were calibrated on – the one whose crop factor is closest to crop wins, matching how a caller would pick by hand; pass 0 to take the first.

out is filled completely, calibration arrays included, and is thereafter independent of db: the handle may be closed and the value stays valid. That is the whole shape of this library – a lens is data, not a handle into a database.

Returns

1 on a match, 0 if nothing matched, -1 on error (including a lens carrying more than LS_MAX_CALIB entries of any kind, which is a database this build cannot represent).

Definition at line 363 of file lensserious_db.c.

References _find_lens_id(), ls_db_lens_by_id(), and ls_db_t::sql.

ls_db_lens_by_id()

int ls_db_lens_by_id	(	ls_db_t *	db,
		long long	lens_id,
		ls_lens_t *	out )

Load a lens by its database id, for a caller that already resolved one.

Definition at line 295 of file lensserious_db.c.

References _db_err(), _fill_calibrations(), ls_lens_t::aspect_ratio, ls_lens_t::center_x, ls_lens_t::center_y, ls_lens_t::crop_factor, ls_lens_t::max_focal, ls_lens_t::min_focal, ls_db_t::sql, and ls_lens_t::type.

Referenced by ls_db_find_lens().

ls_db_lens_fits_mount()

int ls_db_lens_fits_mount	(	ls_db_t *	db,
		long long	lens_id,
		long long	mount_id )

Does a lens fit a camera's mount, upstream's compatibility table included?

Returns

1 yes, 0 no, -1 on error.

Definition at line 411 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_lens_mounts()

int ls_db_lens_mounts	(	ls_db_t *	db,
		long long	lens_id,
		char *	out,
		size_t	out_size )

The mounts a lens is made for, joined with ", ".

Parameters

db	an open database.
lens_id	the lens.
out	caller's buffer; always NUL-terminated when out_size is non-zero.
out_size	its size.

Returns

the number of mounts written, or -1 on error.

Note

These are the lens's OWN mounts, not the mounts it is compatible with. Asking whether a particular camera can take it is ls_db_lens_fits_mount(), which does consult upstream's compatibility table; this is the descriptive string a picker shows.

Definition at line 527 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_lens_name()

int ls_db_lens_name	(	ls_db_t *	db,
		long long	lens_id,
		char *	maker,
		size_t	maker_size,
		char *	model,
		size_t	model_size )

The lens's maker/model, as stored.

Returns

bytes written, or -1.

Definition at line 460 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_lens_range()

int ls_db_lens_range	(	ls_db_t *	db,
		long long	lens_id,
		float *	min_focal,
		float *	max_focal,
		float *	min_aperture,
		float *	max_aperture )

The lens's focal and aperture range, for a picker that lists it.

Parameters

db	an open database.
lens_id	the lens.
min_focal,max_focal,min_aperture,max_aperture	filled in; any may be NULL.

Returns

1 when the lens exists, 0 when it does not, -1 on error.

Definition at line 496 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_lenses_for_mount()

int ls_db_lenses_for_mount	(	ls_db_t *	db,
		long long	mount_id,
		long long *	out_ids,
		int	max )

The lenses made for one mount.

Parameters

db	an open database.
mount_id	the mount.
out_ids	caller's array, or NULL to count only.
max	the length of out_ids.

Returns

how many ids were written (or how many exist, if out_ids is NULL), or -1.

Note

The lens's OWN mounts, without upstream's compatibility table. A fixed-lens camera has a mount of its own and exactly the lenses built into it, and widening that to compatible mounts would answer with lenses that physically cannot be on the camera.

Definition at line 668 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_list_cameras()

int ls_db_list_cameras	(	ls_db_t *	db,
		long long *	out_ids,
		int	max )

Enumerate camera ids, for a GUI's camera picker.

Parameters

db	an open database.
out_ids	caller's array, or NULL to count only.
max	the length of out_ids.

Returns

how many ids were written (or how many exist, if out_ids is NULL), or -1.

Definition at line 564 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_list_lenses()

int ls_db_list_lenses	(	ls_db_t *	db,
		long long *	out_ids,
		int	max )

Enumerate lens ids, oldest-inserted first, for tests and for a GUI's lens picker.

Parameters

db	an open database.
out_ids	caller's array, or NULL to count only.
max	the length of out_ids.

Returns

how many ids were written (or how many exist, if out_ids is NULL), or -1.

Definition at line 436 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_match_lens()

int ls_db_match_lens	(	ls_db_t *	db,
		const char *	maker,
		const char *	model,
		long long	mount_id,
		float	crop,
		ls_db_match_t *	out,
		int	max )

Find the lenses a free-text name most likely refers to.

ls_db_find_lens() answers "is there a lens called exactly this". This answers the question a raw file actually asks, where the EXIF string is a vendor's abbreviation of the name upstream chose – "16-35mm f/4G ED VR" against "Nikon AF-S Nikkor 16-35mm f/4G ED VR", with tokens missing, reordered, or spelled differently.

Scoring is token-based and deliberately simple; see the implementation for what each weight is and why. It is calibrated against liblensfun's own decisions rather than against a specification: tests/match_lensfun.c asks both this and lf_db_find_lenses_hd() the same questions over the whole database and reports where they disagree.

Parameters

maker	may be NULL. When given it is scored, not required – vendors disagree with upstream about their own name often enough that requiring it loses more than it saves.
mount_id	when > 0, only lenses that fit this mount are considered (ls_db_find_camera() supplies it). 0 considers every lens.
crop	the CAMERA's crop factor (ls_db_find_camera() supplies that too), or 0 to ignore it. It is not a tie-break: upstream REJECTS a lens whose calibration sensor is more than 4% larger than the camera's – such a calibration does not cover the frame – and then grades what is left by how closely the two match. Several lenses share a name and differ only in the sensor they were calibrated on, so without this the matcher can return a name-identical lens with the wrong calibration: measured on a Nikon D5300 (crop 1.534), a full-frame row instead of the 1.528 one, and 0.19 px of geometry.
db	an open database.
model	the free-text name to resolve. Required.
out	caller's array, max entries, filled best-first.
max	the length of out.

Returns

how many candidates were written, or -1 on error.

Definition at line 1042 of file lensserious_db.c.

References _compare_num(), _crop_score(), _db_err(), _digest_load(), _parse_focal_range(), _score_tokens(), _tokenize(), ls_tokens_t::bloom, ls_db_match_t::lens_id, ls_db_normalize(), LS_MAX_TOKENS, LS_TOKEN_LEN, ls_tokens_t::n, ls_db_match_t::score, ls_db_t::sql, and ls_tokens_t::t.

ls_db_meta()

int ls_db_meta	(	ls_db_t *	db,
		const char *	key,
		char *	out,
		size_t	out_size )

A meta value by key (built_utc, source, lensfun_db_version, ...).

Returns

bytes written excluding the terminator, or -1 if absent.

Definition at line 691 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_mount_name()

int ls_db_mount_name	(	ls_db_t *	db,
		long long	mount_id,
		char *	out,
		size_t	out_size )

A mount's name.

Parameters

db	an open database.
mount_id	the mount, as ls_db_find_camera() reported it.
out	caller's buffer; always NUL-terminated when out_size is non-zero.
out_size	its size.

Returns

1 when the mount exists, 0 when it does not, -1 on error.

Note

Upstream encodes a real fact in the SPELLING: a mount whose name starts with a lower-case letter belongs to a fixed-lens camera. That is how a consumer tells a compact from an interchangeable-lens body, so the name has to be reachable, not just its id.

Definition at line 641 of file lensserious_db.c.

References _db_err(), and ls_db_t::sql.

ls_db_normalize()

size_t ls_db_normalize	(	const char *	in,
		char *	out,
		size_t	out_size )

The comparison form of a name: ASCII case-folded, punctuation dropped, runs of whitespace collapsed to one space.

Exposed because the importer and the reader must produce identical bytes – the importer stores it, the reader compares against it – so there is exactly one implementation and no locale, ICU version or platform in the middle of it. Non-ASCII bytes pass through untouched.

Returns

the length written, excluding the terminator.

The comparison form of a name: ASCII case-folded, punctuation dropped, runs of whitespace collapsed to one space.

Deliberately ASCII-only and byte-wise: it must produce the same bytes in the importer and here, on every platform, without a locale or an ICU version in the middle of it. Non-ASCII bytes pass through untouched, so a UTF-8 maker name still matches itself; it just does not case-fold, which no upstream name needs.

Definition at line 156 of file lensserious_db.c.

Referenced by _find_lens_id(), insert_name(), ls_db_find_camera(), and ls_db_match_lens().

ls_db_open()

ls_db_t * ls_db_open

(

const char *

path

)

Open a database for reading.

Parameters

path	a filesystem path, or a file: URI. Either way it is opened read-only and immutable; any query parameters the caller supplies in a URI are preserved, but mode and immutable are forced.

Returns

NULL if the file cannot be opened or is not a LensSerious database of a schema version this build understands. Never fails for concurrency reasons: there are none.

Definition at line 87 of file lensserious_db.c.

References _db_build_uri(), ls_db_close(), LS_DB_SCHEMA_VERSION, ls_db_t::schema_version, and ls_db_t::sql.

ls_db_schema_version()

int ls_db_schema_version

(

const ls_db_t *

db

)

Schema version of the open file, or -1. Bumped when the layout changes.

Definition at line 138 of file lensserious_db.c.

References ls_db_t::schema_version.

ls_db_token_digest()

size_t ls_db_token_digest	(	const char *	norm,
		unsigned char *	out,
		size_t	out_size )

Pack a normalised name's tokens into the digest stored in lens_name.tokens.

Layout: uint16 n, then n x uint32 hash, then n x uint8 length. The matcher scores on those alone – it never sees the token text – which is what lets the candidate query read a blob out of a covering index instead of decoding and re-tokenising text on every lookup.

Returns

bytes written, or 0 if out is too small.

Definition at line 794 of file lensserious_db.c.

References ls_db_token_hash(), ls_db_tokenize(), LS_MAX_TOKENS, and LS_TOKEN_LEN.

Referenced by insert_name().

ls_db_token_hash()

unsigned ls_db_token_hash

(

const char *

token

)

FNV-1a of a token, the hash the stored digest and the matcher both use.

Definition at line 783 of file lensserious_db.c.

Referenced by _tokenize(), and ls_db_token_digest().

ls_db_tokenize()

int ls_db_tokenize	(	const char *	norm,
		char *	out_tokens,
		int	max,
		int	stride )

The tokens ls_db_match_lens() compares, from an already-normalised name.

Split on spaces, and again wherever a letter meets a digit – normalisation drops '-', so "16-35mm" would otherwise fuse into one meaningless token instead of matching the catalogue's "16 35mm".

Exposed for the same reason as ls_db_normalize(): the importer writes these into the token index and the matcher looks them up, so there must be exactly one implementation.

Parameters

norm	a name already put through ls_db_normalize().
out_tokens	caller's array of max buffers, each at least stride bytes.
max	how many tokens to write at most.
stride	the size of one buffer in out_tokens, in bytes.

Returns

how many tokens were written.

The tokens ls_db_match_lens() compares, from an already-normalised name.

"16-35mm" normalises to "1635mm" – no. Normalisation drops '-', so the two halves of a focal range would fuse into one meaningless token and stop matching the catalogue's "16 35mm". Splitting where a digit meets a letter, and where a letter meets a digit, keeps "16", "35", "mm", "f", "4g" as separate comparable units.

Definition at line 758 of file lensserious_db.c.

Referenced by _tokenize(), insert_name(), and ls_db_token_digest().