pouët.net

Runtime loading of data

category: general [glöplog]
I'm not sure if this has been mentioned before but:

what is the situation with loading (even saving) data from the disk while the demo is running? is it allowed? I think that while ago it was not - once the demo starts it should not read/write anything to physical discs or the internet.

If this is the case, do you know the reason? what about streaming in video files?
added on the 2011-03-16 18:06:19 by Navis Navis
Streaming from the internet is obviously out for most compos, but why not streaming from disk? Don't remember seeing that rule, and it's been common since at least the amiga days.

Hmm.. then again.. streaming from the internet. We could just write a video player that streams a youtube HD video of the demo, which is how most people watch it anyway so what's the problem? :D
added on the 2011-03-16 18:10:47 by psonice psonice
I have never read a rule which said you could not stream from disk after the demo has started... writing is different though, but reading? Go ahead :-)

I think the reason it's not done much is because given the "usual" needs for a demo, streaming is just not needed (there's enough memory to hold all resources in memory), and streaming would make the whole thing a bit less performant (or much less, if it's not well done).

Streaming from the internet might be another thing though... there's still a bunch of computers which don't have internet (or hi-speed one, for the matter)... I guess streaming from internet and realtime don't yet mix up too well.

Plus, there are limited data plans around (even for landline DSL's), so not sure if people would want their bandwidth to be consumed for watching a demo many times :-)
added on the 2011-03-16 18:13:57 by Jcl Jcl
isnt streaming data from disk commonplace in demos on non-pc platforms already?
added on the 2011-03-16 18:20:32 by smash smash
smash, think so... i remember many amiga demos which had the floppy disk almost constantly spinning (i "guess" streaming... but i have no clue on how amigas were programmed so that's just a wild guess)
added on the 2011-03-16 18:22:01 by Jcl Jcl
As has already been said, streaming from disk is usually permitted. Writing to disk is often not permitted, but I'm not really sure why.
added on the 2011-03-16 18:31:19 by kusma kusma
kusma: i guess that's a rule from the old times where people would make demo compilations on CD (as exe's), which has never been modified since :-)
added on the 2011-03-16 18:37:52 by Jcl Jcl
... and from when there were fears of small demos writing datafiles for other demos to overcome the size limits in party compos :-)
added on the 2011-03-16 18:39:12 by Jcl Jcl
ah ok. I think I got this rule in my head from the days of CDs: that the assumption of a very fast read from a disk was not always correct. That would kill your framerate.


Accessing the internet is a different thing altogether, and I'm sure some firewalls would start moaning halfway through watching, thus spoiling your experience.
added on the 2011-03-16 18:41:10 by Navis Navis
kusma: i suppose it cant be (or usually isnt) enforced, so i suppose it's more a rule to discourage it from accidentally fucking up the compo machine.
added on the 2011-03-16 19:13:46 by Gargaj Gargaj
Reading/streaming from disk is no problem and at least BP rules always allowed it (not sure about other parties).

Writing is slightly dicey in that you'd prefer the machine to be in the same state before and after running a demo (just to avoid potential problems when running it a second time), but unless you're stupid about it that's not a problem and BP generally allowed it.
added on the 2011-03-16 19:38:53 by ryg ryg
Psionice: Other Nice idea.. presenting a realtime demo in a youtube HD video layout design ;-)

(something like Andromeda did in the early 90's presenting a realtime demo in an amiga dos layout)

added on the 2011-03-16 19:42:07 by magic magic
Quote:
Writing to disk is often not permitted, but I'm not really sure why.
It stems from the days of CD-ROM compilations.
added on the 2011-03-16 19:45:22 by gloom gloom
I don't see a reason for a demo to write anything down. Maybe selected gfx settings for future runs or performance details for benchmarking, but what else?

I kind of liked the idea Jcl presented, though - copy all the data you want from a large demo to some obscure location and win the next 4k compo by doing amazing amounts of stuff :) A pretty fragile plan, though, as the larger demo won't probably be run on the newer compo machine - and then there are the ethical downsides.
added on the 2011-03-16 19:55:40 by msqrt msqrt
it was and is allowed at buenzli, no problem with streaming data from disk. it's just that most coders are too lazy to stream properly and waste tons of memory. but in the end, who cares? it is just the demo running, and 90% of the demos don't do magic things and eat up those neat 4gb/1.5gb vram or more in todays machines. oh wait, there are PCs with less memory? but hey, those PCs in the end normally don't have the necessary gfx power because everyone is wasting fillrate, so who cares? and yep, demos are watched on youtube today... (sad thing.)

writing on disk: i encountered that many 64ks 10yrs ago did unpack some files (like xms for minifmod) to disk... or there are demos which unpack to disk because of general lazyness. or write stupid logfiles. (sad thing aswell.)

oh yes, and i did demos that write to disk. and i watch demos on youtube. and i think i never did proper and good streaming, so blame me aswell :)
added on the 2011-03-16 21:00:01 by prost prost
Quote:
I kind of liked the idea Jcl presented, though - copy all the data you want from a large demo to some obscure location and win the next 4k compo by doing amazing amounts of stuff :) A pretty fragile plan, though, as the larger demo won't probably be run on the newer compo machine - and then there are the ethical downsides.

And that's why we used a separate (but identical) machine for compo preparation on BP, and show the smallest compos first. :)

You could still submit a bunch of junk 4ks that dump a few kilobytes of payload each to extend the size limit for 64ks. :) Preventing that involves wiping the compo machines from a clean image after every compo, which is possible but extremely inconvenient and unlikely to be done in practice :)
added on the 2011-03-16 21:20:07 by ryg ryg
Ryg, thanks for the useful adv.. information :)
added on the 2011-03-17 00:09:07 by msqrt msqrt
PC demos streaming from disk? Considering that PC demo file size limits were at about 1/16th of the compo machine's RAM for the last 10 years this didn't make any sense anyway. Just map your packfile into memory and you're set (especially on 64bit systems where there's absolutely no reason anymore NOT to use mmio for read only access). Also, in most cases this is faster than any fancy loader code you could hack up.

To top it off, the graphics APIs on Windows weren't exactly streaming friendly (as in: lots of overhead for resource loading and strictly single-tasked) before DirectX11, so I doubt anyone bothered. Perhaps people streamed sound data, but then again, loading a 10 megabytes blob doesn't hurt at all, and the abovementioned mmio gives you "streaming" for "free".

As for writing to disk: There are certain intros or demos that create a cache for all the generated stuff on disk because it's just so much faster to read it back than generate it again (because nomen est omen). That's the one thing I can think of in terms of writing to disk, and it's perfectly ok, even in demo compos (organizers hate long loading times :D).
added on the 2011-03-17 02:18:56 by kb_ kb_
Quote:
copy all the data you want from a large demo to some obscure location and win the next 4k compo by doing amazing amounts of stuff

...and done
added on the 2011-03-17 03:34:57 by Gargaj Gargaj
ryg: Couldn't something like Sandboxie be used in theory to prevent applications from writing to disk? I doubt it'd ever be worth the hassle though, I can't remember having seen a cheat like that myself before.
added on the 2011-03-17 10:05:15 by kusma kusma
Hm, I guess I should have read Gargaj's post first :(
added on the 2011-03-17 10:06:12 by kusma kusma
Yeah, writing isn't the problem, reading is. But there are tools you can use to easily detect both.
added on the 2011-03-17 10:18:00 by doomdoom doomdoom
Quote:
(i "guess" streaming... but i have no clue on how amigas were programmed so that's just a wild guess)


"Streaming" is a good word for it, yes.
added on the 2011-03-17 10:23:42 by doomdoom doomdoom
Ryg: at work some time back we used a package called 'deep freeze'. You build your basic install up, then freeze it. From then on, whenever it's rebooted it gets restored to the initial state, anything new or changed is removed.

Can't remember the details too well now, but I remember it being pretty fast, definitely nothing near re-imaging. Of course you'd still have a reboot between demos, but maybe with a fast system + ssd it's doable (and probably desirable incase one demo fucks up something in the system, spoiling the next demo).
added on the 2011-03-17 11:00:15 by psonice psonice

login