compendium on extrapolation - wip iykyk
beyond pigment & pixel (pigment -> pixel -> 🗯)
prognose, trend, forecast, estimation, predict, future

extrapolation explained by Johan Rijpma

first prototype:

realtime

0324_2d_linear realtime

18.03.

some (reverse)extrapolation combinations. no pre no post.

rgb + irr 14

rgb + uv 06

rgb + xray 08

rgb + xray 15

irr + uv 12

uv + xray 08

uv + xray 13

uv + xray 20

reverse extrapolation uv -> cam

rgb + uv 01

rgb + uv 04

rgb + uv 07

rgb + uv 08

rgb + uv 09

rgb + uv 11

rgb + uv 12

rgb + uv 13

rgb + uv 15

rgb + uv 20

extrapolation cam -> uv

6 works. each 4 different captures -> 6 combinations. 20 snaps per combination = 120 per work
6 * 120 = 720ed.

blur + segmentation as pre?

kmeans, 30%,

kmeans, 100%, 15blur, 30 cluster

19.03.

js webgl experiments

js/0319_pipeline/index2.html

5 clusters

15 clusters

30 clusters

60 clusters

20.03.

sandwich extrapolation geometry

realtime

24.03.

scale/closeup test (segmentation only)

500x500 30 clusters

1000x1000

2000x2000

500x500 30 clusters

1000x1000

2000x2000

500x500 20 clusters

1000x1000

2000x2000

FFT frequency spectrum modulation
The Fast Fourier Transform (FFT) and Inverse FFT (IFFT) serve as fundamental mathematical operations in computer vision that facilitate conversion between spatial and frequency domain representations of images. The FFT decomposes an image into its constituent frequency components, producing a complex-valued representation where the central region corresponds to low-frequency information (representing the overall image structure) while peripheral regions encode high-frequency details (capturing edges, textures, and noise). The IFFT performs the complementary operation, reconstructing the spatial image from its frequency representation. This bidirectional transformation enables numerous practical applications in image processing, including frequency-selective filtering, pattern recognition, image compression, and noise reduction, as specific frequencies can be isolated, modified or suppressed before reconstruction. These techniques prove particularly valuable when certain image features or artifacts manifest more distinctly in one domain than the other, allowing for more effective manipulation and analysis.

orig inv. uv

⭢

fft

⭢

blurred fft

⭢

inv. fft

orig inv uv

⭢

fft

⭢

directional filter

⭢

inv. fft

orig inv uv

⭢

fft

⭢

directional filter

⭢

inv. fft

orig inv uv

⭢

fft

⭢

directional filter

⭢

inv. fft

25.03. - fft, directional filter, binary, closing, dilating, segmenting, crop scale

analyze images to decide next processing steps of the future pipeline

1- dame_in_weiss_reverse_interpolated_and_spiral_convolve_and_clustered_by40
2- dame_in_weiss_reverse_interpolated_and_median7_and_clustered_by30
3- dame_in_weiss_4286_auflicht_irr_extrapolation_007_4clusters
4- dame_in_weiss_reverse_interpolated_and_blurred_and_clustered_by30

1000x1000 random crops:

computing fft on all three channels? fft only on brightness channel?

directional fft filter

the following are directional fft filtered, binaried, dilated and then k-means segmented images cropped and scaled up.

50x50 -> 2000x2000

100x100

200x200

300x300

600x600

no jpeg compression artefacts

0325_directional_fft_binary_dilation_segmentation_overview

just cropped and scaled of extrapolation without postpro

26.03. pipeline v1
res:2000x2000

without kmeans:

workflow: k-means on low-res cropped image. then scale to 2000x2000

27.03. V1

v1. params: cropsize, clusters, spatial

28.03. v1 final

ffmpeg -framerate 30 -i %07d.png -vf "scale=500:500" 0328_v5.gif

v1 realtime

reading:
https://www.workingtheorys.com/p/make-something-heavy
https://www.e-flux.com/journal/10/61362/in-defense-of-the-poor-image/
https://pixlpa.substack.com/p/elements-between-domains

01.04.

cycling without a second image. testing to see if this is another vector. but it isnt

various zoom-level sectors

w/ focus on same area

animation in centerpos and extrapolation factor

05.04 - animation, colors, multi-setup

anim: extrapolation, position, zoom, spatialweight.
random: color palette (2-16 colors), kmeans 80%, 6 paintings, each 6 combinations

16.04 - v2.3

v2.3

interaction:
click to zoom into quadrant

keys:
s: save png
i/o: start/stop video capture
r: random parameters
k: random anims
m: stop anims
v: extrapolate anim
b: spatial weight anim
c: zoom anim
x/y: x/y anim
1-6: random image pair of painting #1-6

click bottom green text copies "hash" to clipboard
ctrl+v applies a "hash"

edge detection on 4th quadrant

29.04. interface research & UMAP

simple umap via pca on 600x600px imgs

100 images, n_components=70

600 images, n_components=100

1500 images, n_components=100

05.05. concept from transcript

EXTRAPOLATION: Venturing Beyond the Visible
This generative art project explores the speculative realm that exists beyond the known dimensions of Gustav Klimt's masterpieces. Working with multiple imaging techniques (standard photography, infrared, ultraviolet, and X-ray) of Klimt's paintings from the Belvedere Museum, I create digital artifacts that extend beyond the visible spectrum into imagined territories.

The Speculative Nature of Extrapolation
Extrapolation, by its very nature, is an act of speculation – a mathematical and philosophical venture into the unknown based on established patterns. Where interpolation connects known points with reasonable certainty, extrapolation reaches beyond the boundaries of what we can directly observe, proposing what might exist if patterns continue. In this project, I embrace the inherent uncertainty and creative potential of extrapolation as a conceptual framework.

Why Two Image Layers?
The power of using two different image layers (such as standard photography and infrared or X-ray and ultraviolet) lies in the contextual relationship they create. A single image transformation might produce interesting effects, but lacks the meaningful dialogue that emerges between two perspectives of the same subject. By extrapolating between two different imaging techniques, patterns emerge that reveal something about their relationship – how structure informs appearance, how underdrawings influence final composition. These different imaging techniques each reveal distinct aspects of Klimt's process – the visible surface that audiences have known for a century, the preliminary sketches visible under UV light, the structural elements revealed by X-rays. By algorithmically extending beyond these known perspectives, this work speculates on another dimensional layer of these paintings, one that exists only in the digital realm but is directly derived from the physical artwork.
The resulting compositions invite viewers to reconsider what we mean by "seeing" an artwork completely, suggesting that even our most advanced imaging technologies capture only specific facets of the whole. Through digital extrapolation, we can imagine new ways of engaging with historical art that honor both preservation and reimagination.

25.05.

Hommaga à Klimt
Out of Bounds
Out of Distribution
A scanner darkly
Spectral Extrapolation

07.06.

Sucher
View Finder
Science Fiction