pouët.net

I thinking in feeding a raw file to the GPU and play with some shaders. Has anyone here done something similar?
I'm using LibRAW to decode the RAW file to an array of something more useable.

not that I can help with the question, but.... I don't understand how one should _decode_ a _raw_ image. Isn't a raw image just... raw?

iq: no. I suppose xernobyl means raw from reflex cameras. These usually are propietary and closed formats and/or not as raw as you would expect sometimes.

or he might actually simply mean raw uncompressed data from digicams (e.g. turning off compression removes the possibilit of compression artifacts).

I think he means raw from DSLRs, so what texel said.

I'm doing this actually - I use the quicktime framework to dump raw images onto the gpu as 16/32bit float images. No idea if this works on windows - quicktime framework is available, but maybe it doesn't include raw support the same as on osx.

Stuff you need to watch out for:

Colour profiles. If you load the raw data directly the colours will be really inaccurate (this is part of what the 'normal' raw conversion software takes care of). If that matters, try to get hold of the colour profile and apply it at the end of the processing chain, or write your own colour curves stage (this is what I did).

Texture limits. This really fucked me over at one point.. my gpu supports 4k*4k textures, so I figured it would be enough for my purposes. Well, it's not 4k at all, it's actually the equivalent amount of vram for a 4k*4k 8bit image. So 2k*2k for 16bit, or 1k*1k for 32bit. I need 32bit, and 1k is useless :( (I'm doing tile based processing to get round this now..)

I mean doing your own debayering.

I have pure greyscale 12bits RAW data direct from the sensor. Now I'll try some simple linear interpolation of the bayer pattern to get some color. I'm not really sure where I'm heading with this.

Using the GPU for this kind of stuff is certainly possible. It's what's being used by apps like SpeedGrade to do realtime debayering.

Yeah, I can't see a problem with debayering on the gpu. Are you doing video or just stills (from .raw files)? At some point I might have to deal with video in a raw-like format (4k*4k sensors too.. luckily at very low framerates ;) so let us know how you get on.

One thing you might want to look at btw: if you have say a 1000 pixel sensor, the raw data is effectively at higher resolution (2000) because the colours are offset. You can use super resolution here to enhance the resolution, especially if you have video.

Just stills.
There is no way to get RAW video from my camera. I've read some things about using super resolution in video in relation to the bayer pattern. It makes sense, because the image doesn't move from green to green, but from green to red to green to blue... Stupid firmware doesn't do that :(
I'm not sure what I want to do actually. It started from a discussion about digital black and white.

Hmm.. digital b+w is interesting. You could have 'almost' 4x resolution quite easily, just by using the native resolution of the raw file and infilling the colours between pixels. You'd have some fringing, and perhaps some colour speckle where detail exists in one channel but not in others, but you could fix that mostly with some cunning shader magic :)

That's how the RED camera gets its magic "4k" resolution from. Sure, it's a 4k sensor, but a bayered one. ARRI's cameras also have a bayered sensor, but they have the good sense to call it 2k resolution.

People don't care though, because 4k is "more" and "more" is "better" w00t RED RULES

If they're doing something basic, I'd call it 2k+. If they're doing some super resolution magic, I'd take the '4k' any day :)

http://www.thedailynathan.com/demosaic/
That site is interesting.

That pic I posted has the RAW image on the left (converted to an 8 bit PNG), the raw with the bayer pattern applied on each pixel, and a highly saturated photo from Nikon's NEF driver.

I can't find any info on demosaicing to black and white. It shouldn't be too complex. There's probably something better than interpolating and mixing each RGB channel.

psonice: It's not super resolution sadly. It's tricky enough getting enough FPS out of a CMOS sensor as it is :)

sagacity: the fps would be the same, super resolution is done in software later.

The super-resolution for b&w is not possible as you might be thinking about.

If you have, a, let say, 6 mp camera, you don't have 6 million bayer patterns, but only 1.5 million bayer patterns.

So, the b&w algorithm should be like this:

- Use the best as possible demosaicing algorithm for colour. The best one depends on your image.
- Desaturate the image.

So, what you get is a 6 mp b&w image, and nothing more. So, only in really extreme cases you are going to do something better than just desaturating the image you got from the camera's demosaicing algorithm. One of these extreme cases would be a scene full of white/grey/black objects only with only full white light. Even in that case, you don't have more resolution as of megapixels, but maybe a sightly sharper image (and probably with more noise).

Psonice: Oh, but then it's just interpolation. I thought you meant super-resolution in the sense of grabbing multiple slightly offset images and using that to make a bigger image.

texel: so the 6mp camera effectively has <6mp resolution? Hmm.. that's a bit shitty! Well, it would still be possible to do super resolution and get that 24mp image, but it would no longer be trivial to to :)

sagacity: there are many methods for super resolution, some use a single image rather than multiple. Multiple image methods tend to work better though.

Isn't Super Resolution just for video?

No, stills too. It's easier with video, that's all.

Damn you, anti-aliasing filter!
From what I see anti-aliasing filters do more harm than good.

anti-aliasing FILTER?