squishy by Logicoma [web]
screenshot added by ferris on 2020-03-29 21:30:41
platform :
type :
release date : march 2020
  • 29
  • 1
  • 0
popularity : 62%
  • 0.97
alltime top: #3561
added on the 2020-03-29 21:30:41 by ferris ferris

popularity helper

increase the popularity of this prod by spreading this URL:

or via: facebook twitter pinterest tumblr


been a long time coming :) feel free to contact if you have any issues/questions (especially this close to revision!)
added on the 2020-03-29 21:32:27 by ferris ferris
Thumb up!
rulez added on the 2020-03-29 21:48:02 by ham ham
power to the people!
rulez added on the 2020-03-29 21:49:31 by v3nom v3nom
you are all cheaters! ;)
rulez added on the 2020-03-29 22:03:44 by HellMood HellMood
Finally! :)
rulez added on the 2020-03-29 22:05:58 by cce cce
Crashes on several exes.
added on the 2020-03-29 22:06:28 by hitchhikr hitchhikr
hitchhikr: bug reports welcome, especially with more detailed info than that :)
added on the 2020-03-29 22:08:22 by ferris ferris
OMG... O M G...
rulez added on the 2020-03-29 23:06:32 by alkama alkama
My bad, it just doesn't work on my main computer because the packer & depacker use unsupported simd instructions, works perfectly on my laptop.
rulez added on the 2020-03-29 23:07:05 by hitchhikr hitchhikr
rulez added on the 2020-03-29 23:33:16 by slerpy slerpy
ooh squishy!
rulez added on the 2020-03-29 23:51:06 by wysiwtf wysiwtf
hitchhikr: cheers, thanks for testing further. There's likely still bugs tho, so if you find any lmk!
added on the 2020-03-30 00:12:44 by ferris ferris
w00t! can't exactly test it tho
rulez added on the 2020-03-30 00:15:30 by porocyon porocyon
puni puni~
rulez added on the 2020-03-30 00:44:23 by noby noby
Does my bum look big in this?

rulez added on the 2020-03-30 00:53:15 by djh0ffman djh0ffman
rulez added on the 2020-03-30 01:04:30 by gopher gopher
rulez added on the 2020-03-30 01:31:15 by psenough psenough
rulez added on the 2020-03-30 01:51:06 by cpdt cpdt
tool thumbs!
rulez added on the 2020-03-30 04:26:14 by sensenstahl sensenstahl
yeah, gonna test this. output=compo-winnar... =)
rulez added on the 2020-03-30 16:16:27 by chlumpie chlumpie
I tried it on a few random exe files but it either didnt do anything at all or breaks with some "chunk too large" error.
I get that this is targeted at 64ks but out of interest: what is the maximum filesize this works on?
added on the 2020-03-30 16:27:23 by wysiwtf wysiwtf
wysiwtf: I searched "too large" in the repo and the only error I can find is this: "Compressed size too large; can't adjust image base to make room for compressed image." Is that the error you're seeing?

You'd need quite a large compressed image and/or a low base address (it should probably be mentioned that you'll need a fixed base address, eg. `/DYNAMICBASE:NO` for MSVC) for that error to occur. The unpacker/headers/etc is placed beneath the original image in memory, and if the compressed size is larger than the image base, that's not possible. The maximum filesize uncompressed can be quite large, but if the compressed image is much more than 64k I wouldn't make any promises. It's probably safe up to 128k judging by the adjustment code but I haven't tested it very rigorously. It's quite likely that (de)compression is going to be horribly slow at this size anyways.
added on the 2020-03-30 16:43:45 by ferris ferris
I've added a ticket to add a check/error for dynamic base address btw.
added on the 2020-03-30 16:48:16 by ferris ferris
Also "didnt do anything at all" is surprising - I would at least expect some kind of panic. Feel free to send me some test exe's (to yupferris/gmail) if you think I should have a look.
added on the 2020-03-30 16:54:00 by ferris ferris
rulez added on the 2020-03-30 17:14:36 by SiR SiR
tool thumb
rulez added on the 2020-03-30 19:45:07 by yzi yzi
what yzi said
rulez added on the 2020-03-30 23:50:56 by wullon wullon
Not sure what to say.

Immediately throwing it to the public makes it tons more prone to being detected, although AVs will complain regardless even about plain NASM/FASM executables being assembled. I just though given up on trying to cooperate with those AV companies since their rules on detection are so bizzare.

Technically its nice. I guess you are using all that SIMD for ANS with some LZ77 using suffix sorts or something.

I'm very glad you lit a fire under my ass to keep working on mine.
rulez added on the 2020-03-31 00:31:08 by mudlord mudlord
wysiwtf: I searched "too large" in the repo and the only error I can find is this: "Compressed size too large; can't adjust image base to make room for compressed image." Is that the error you're seeing?
yep thats the one, sorry for being inaccurate.
the ones where it didnt do anything at all were intro files which were already compressed in some way, for example the fr041-debris.exe. there was no output and no panic, it just didnt do anything.
for "random exe files" i tried with the doom eternal exe which is close to 70mb and the openttd.exe which is close to 1mb, always getting the error you mentioned above (but since the output wouldnt possibly be near 64k that makes sense, probably).
added on the 2020-03-31 02:02:50 by wysiwtf wysiwtf
wysiwtf: cheers, thanks for clarifying. Yeah the already-compressed exe's won't work due to header packing already done by other packers, though I'll add a ticket for reporting a proper error in that case. As for the other exe's, the above comment applies :) thanks for testing!
added on the 2020-03-31 09:59:45 by ferris ferris
I guess you are using all that SIMD for ANS with some LZ77 using suffix sorts or something.

Actually, all that SIMD is for mixing predictions. Similar to later PAQ/kkrunchy there's a two-stage mixer that mixes the 72 (data section) or 135 (code section) context model predictions with 7 different weight sets (selected by additional contexts) in the first stage, and then it mixes those mixed outputs in a second stage with a single weight set. Both stages are processed 8 elements at a time; that's where the SIMD is (and I think the newest instruction I used was phaddw, which is just SSSE3; I did experiments with both more lanes and more precision and neither gave compelling results).

As for ANS, the main benefit there is, as you (indirectly) suggest, that you can mix different bitstreams naturally and be able to (de)code them in parallel. However this relies on the model being able to keep up (typically by updating more sparsely, and predictions need to be parallelized as well), and with a model this heavy (and certainly memory bound), there's no way we're doing any of it in parallel, so then we're left with a single stream. In that sense, ANS doesn't offer anything compelling at face value (though I did compare both an ANS decoder as well as the arithmetic one I went with just in case it had better precision or something, but neither really outperformed the other, and the arithmetic decoder was one instruction smaller and didn't require encoding the data backwards like ANS does!). And the whole multiple streams thing is a speed optimization that you typically trade some compression ratio for and, to my knowledge, none of the "max ratio" compressors are doing that simply because you get better compression performance with per-symbol model updates anyways (and we can afford this because we're only working with a few hundred KB of actual data here).

So yeah, no LZ/ANS shenanigans this time around (I have a C64 4k packer that actually does use that stuff if anybody wants to give it a test run!), just straight-up big, beefy context mixing. :)

I'll have many more gory details on my talk at Revision in a couple weeks btw - check the schedule/timetable for the time as it gets closer :)
added on the 2020-03-31 10:44:42 by ferris ferris
rulez added on the 2020-04-01 08:41:35 by sagacity sagacity
Cannot wait for your talk :D
added on the 2020-04-03 04:25:25 by mudlord mudlord
Just did a quick update release to add some helpful usability things: version 0.1.1 (05.04.2020)

- Added several command line options; see usage for info.
- Added warnings for resources, as they can be somewhat large.
added on the 2020-04-05 22:05:10 by ferris ferris
Awesome presentation/seminar & thanks for taking the time to answer our questions ;-)
rulez added on the 2020-04-12 22:33:10 by fbrz fbrz
love crunchers
rulez added on the 2020-04-12 22:37:55 by roudoudou roudoudou
really nice seminar, I think I understood everything up until a moment, then it just fell apart when you went more in depth into the models, I was doing some cooking while watching it though, so maybe a re-watch would help.

I was curious what soft of OS and low level hacks go into one of this since I'm somewhat familiar to that area. That was answered and it was a lot more gentle and well behaved than expected.

I'm not very familiar with compression except for the high level theory, so might need to work on that.
added on the 2020-04-13 01:58:34 by musk musk
oh, almost forgot about the upvote... or maybe I should wait until I try it? nah, upvote well deserved
rulez added on the 2020-04-13 01:59:58 by musk musk
version 0.1.2 released (22.04.2020)

- Reworked jmp/call transform to be driven by the model's state-based disassembler, which allows the transform to more accurately identify these instructions. This provides better compression while also fixing compatibility issues when the previous inverse transform would produce invalid code after decompression.
- Adjusted indirect models slightly for better compression.

These changes gain anywhere from 50-500 bytes on each intro I tried, so have fun squeezing another scene or two into your next intro ;)
added on the 2020-04-22 20:44:19 by ferris ferris
will use on next 32k prod
rulez added on the 2020-06-18 18:52:15 by NR4 NR4
Making a 64k in 1995: executable is 100k after putting in setup code.

Making a 64k in 2021: ok done sheesh I still gave 10k left how the hell am I going to ever fill this.
rulez added on the 2021-01-06 08:01:53 by pestis pestis
version 0.1.3 released (07.02.2020)

- Fixed a possible code section unfilter regression introduced in 0.1.2, which affected a handful of intros.
- Fixed an issue where certain large resources could get corrupted during packing.
- Added additional 32-bit executable sanity checks which should catch most invalid/malformed input files.
added on the 2021-02-07 18:11:46 by ferris ferris
erm, that was supposed to say 07.02.2021 :)
added on the 2021-02-08 12:40:49 by ferris ferris
nice ux and compression rates.
i like the name :D
rulez added on the 2021-04-07 13:00:28 by wrighter wrighter
version 0.2.0 released (23.03.2021)

- Migrated to x86-64. Note that the included README claims that all further squishy releases as of 0.2.0 will only target x86-64, and no longer support x86. However, x86-x64 proves to be a bit more difficult to compress than x86, so this was a bit hopeful. Expect further releases to support both x86 and x86-x64. In the meantime, previous versions are still available if x86 is desired.

Note that while I made this release public now, it's been in the wild since March for those who have asked for it/been brave enough to test it, and it was even used for at least one prod from Revision this year (that I know of) - hence keeping the original release date from the archive.
added on the 2021-07-02 13:50:49 by ferris ferris
rulez added on the 2022-01-17 16:53:46 by Anat Anat

submit changes

if this prod is a fake, some info is false or the download link is broken,

do not post about it in the comments, it will get lost.

instead, click here !

[previous edits]

add a comment