pouët.net

Digitally signing kkrunchy compressed releases

category: code [glöplog]
or well "virus" i mean :)
added on the 2015-04-23 12:39:17 by maali maali
Quote:
teo: that would still flag the .zip as malicious by some very proactive scanners, as the virus is in the .zip


there is a way round that.

use password protected zip files, and share a known password.

we do this all the time even when firing legit code signed binaries around by email as some mail systems block binaries full-stop.
added on the 2015-04-23 12:42:53 by Canopy Canopy
Quote:

I wish everyone who says people should uninstall av software their own ransomware incident so they shut up...

Not so nice... If shit happens, shit happens.
If you want to use some random anti-virus software I wont tell you not to do so. This should also be valid the other way around.

As Kylearan (and I - between the lines) mentioned - uploading demoscene binaries to virustotal.com is close to the worst idea ever. That's how you create false positives.

We had only a few comments on mercury releases mentioning (false) virusscanner positives.
I don't care too much - I do that stuff for the fun it and not for brainless consumers who want to get their daily dose of demos.

Creating an awareness about the issues with false positives wouldn't be a too bad thing IMHO.
Would certificates really help at all? I mean, if some vendor randomly blacklists binaries packed with kkrunchy, it would still happen anyways - you would still have to ask for white-listing, waving around your cool certificates...

Sad fact aside, most people will nowadays watch your intro on youtube anyways.

For the paranoid ones: Maybe we could add an optional human readable state of the art checksum to pouet uploads - so the original authors of a release can at least give the user the opportunity to check whether one downloaded the right thing?
(Would introduce a lot of new issues: e.g. How the check whether the checksum comes from the original authors? etc.)
added on the 2015-04-23 13:11:54 by las las
I think this is a good and interesting idea, which would assist with scene outreach, because:

1) YouTube captures cannot always successfully replicate the look of a demo;

2) scene.org, pouet.net, and demozoo, as they are under scener control, are more reliable repositories for scene content than YouTube, as someday the content there will go poof, possibly without backups made unless ArchiveTeam or a similar group is feeling particularly masochistic;

3) Because the scene was historically associated with cracking, demo files that set off virus filters may cause credibility issues for the modern scene and actively hamper outreach, which is worth doing even to less clueful people as they can become more clueful someday.

So a solution would be good. I support a centrally-administered solution for several reasons, although as is often the case I lack the expertise to provide useful support (sigh).

Reasons being:
1) Credibility
2) Staying power
3) Consistency

At least one person on this thread has mentioned a solution from work as something that might be useful in this instance.

Who in the scene would be particularly knowledgable about affordable, effective, and elegant file verification? Alternatively, which sub-industries within IT might have had to solve similar problems?

With apologies to the X Files . . . The answer is out there. (:
Quote:

use password protected zip files, and share a known password.

we do this all the time even when firing legit code signed binaries around by email as some mail systems block binaries full-stop.

This does not always help in case the password is only used to encrypt the files, but not the filenames.

I observed mails to be dropped with 0 byte dummy.exe, zipping with and without password didn't help either. IIRC password protected rar worked, but I'm not certain. You have also to encrypt the filenames in order to be "sure".
added on the 2015-04-23 13:30:28 by las las
exe packing almost always comes with false positives no matter what. even if the exe is signed, there are avs that will still flag it as a virus in case they don't like the code structure or the actions the code does in the system (e.g. creating a handle - locking a buffer, cleaning a memory location, decompressing into memory, etc.), several avs even flag undesirable programming languages as viruses as well, (happened to me once or twice) so false positives are unavoidable for small exes were code has to be optimized to the byte. also kkrunchy has been used in the past to pack a virus and since it is used by a small community like the demoscene they don't bother to add a kkrunchy decompressor ( like it happens with upx, for instance) into the avs code to scan the file properly, thus flagging everything packed with it as a potentially unwanted (or virus) file. i would propose to have a link to a disclaimer page right next to the 64k, 8k, 4k..., category on pouet prod page for the sanity of the newbie user (this prod may cause a false positive alarm - don't be afraid to download) and also include the unpacked exe to a different zip and link, of the packed one for those too afraid of the compressed exe.
added on the 2015-04-23 13:36:48 by Defiance Defiance
zips including checksums might be a decent starting point to avoid certification bullshit. doesnt completely solve the issue but atleast gives the user some assurance the file has not been tampered with .

although to my knowledge only security paranoid users actually check checksums, but well... better then nothing.
added on the 2015-04-23 14:17:29 by psenough psenough
having a message explaining antivirus alerts on pouet and providing alternate unpacked versions of the most popular intros also sounds like good ideas to educate and bypass the AV issue.
added on the 2015-04-23 14:19:42 by psenough psenough
Quote:
? Alternatively, which sub-industries within IT might have had to solve similar problems?


I work in err.. encryption/data security but we are very closely linked to a large AV outfit.

However. everything official either at the windows side involves normal code-signing (two types, one for apps., one for drivers) the other is for uefi/secureboot which requires co-signing by M$ and code etc has to be reviewed by them before they'll sign it.
added on the 2015-04-23 14:38:58 by Canopy Canopy
Quote:
As Kylearan (and I - between the lines) mentioned - uploading demoscene binaries to virustotal.com is close to the worst idea ever. That's how you create false positives.


Yes, it is. You know it and I know it, but virustotal still stands as a bastion of "making sure" where even I upload any shady files I download before I run them. Most people who do this are not demosceners, and THEY don't know. And so this highly limits the amount of outreach possible, especially when you look at the original results. And when your website is unreachable from various places because it's been blacklisted by an overzealous av company it gets even more personal. This stance of 'just let these people enjoy their ignorance' is not something I share, because without reaching people 'not in the know' you can't get more demosceners (and possibly more people to compete against in an awesome 64k compo like the one we just had). I also prefer that people watch our productions in real-time, and I make every effort so that as many people can do that without issues as possible.

On the other hand, I'm not sure how I feel about distributing uncompressed releases. It kinda removes the magic when you see that the prod is originally 800k :D But even compressing with UPX will get it flagged by some av packages, so replacing kkrunchy in larger, "safe" versions isn't an option either. Maybe a "do it yourself" package would work that contains both the uncompressed release and the packer, but I feel that the whole code obfuscation aspect is also a part of the allure of size limited productions, which would be lost this way. And yes, google filters even encrypted zip files if they have an exe in them, they need to be renamed.

At any rate, this discussion is now at least possible with the changes I posted, which is a step in the right direction I think.
added on the 2015-04-23 16:41:40 by BoyC BoyC
Yes, you cannot prevent your binaries from getting uploaded to virustotal & the like.
As soon as some random DAU or random organisation/company/provider admin/"security" faggot receives a warning from his security simulation suite he might backcheck using it anyway.

Some AV stuff even reports programs like pskill or alink as malware, thus trying to create a 100% av compatible file is likely to fail sooner or later anyway.
Also, it looks like that the AV vendors do not care at all if you inform them about false positives unless they´re from some major player.
added on the 2015-04-23 17:42:41 by T$ T$
Quote:
Also, it looks like that the AV vendors do not care at all if you inform them about false positives unless they´re from some major player.


That's not exactly true.
Original state vs Current state
Most of them listen, but it's tedious to reach them. But yeah there are some who don't give a damn, mainly the ones remaining on the list. The problem is that any fixes to the prod need to go through this whole thing again, which is why I'm looking into this at all to see if there are any ways to get around this.
added on the 2015-04-23 17:56:27 by BoyC BoyC
All this looks like a lost battle but I wish you good luck.

Distributing not-so-packed versions (perhaps with only data packed the AV would be happy?) seems to be the best workaround.

By the way, this thing is exclusive of Windows or can also happen on linux?

Malicious question, I know. :D
added on the 2015-04-23 18:22:36 by ham ham
It's bad enough to explain what 64k is to someone, imagine when the file is not 64k. "Yeah all you have to do is run it through this other EXE from the command line and now your virus scanner will delete it."

Screw that.
added on the 2015-04-23 18:26:11 by Gargaj Gargaj
Quote:
We had only a few comments on mercury releases mentioning (false) virusscanner positives.

Maybe it's because ours are more popular than yours.

BB Image
added on the 2015-04-23 18:52:05 by zoom zoom
Zoom. Flawless victory.
added on the 2015-04-23 19:08:17 by gloom gloom
The solution to not-sharing the certificate would be "easy" (in quotes)... instead of having a service to sign the "kkrunchied"-executables, have a service where you send the unpacked exe, it checks it (on virustotal or sumthing), and if it comes out clean, then it packs it (with a chosen kkrunchy command line) and returns the packed+signed exe.

There's still a problem on who would pay the for the certificate... and again, if the service somehow gets some malware through and it gets "caught" by antivirus, the certificate would probably get blacklisted by AVs (and you may even have problems with the CA).
added on the 2015-04-23 19:15:26 by Jcl Jcl
Jcl: I like that idea, it could work, combined with some kind of access-control to make sure that any random person from the internetz can't use it to sign anything they like.
added on the 2015-04-23 19:50:36 by BoyC BoyC
BoyC, as I see it, instead of public open access, have scene.org (or some other) host the service and let only access to verified party organizers or something (maybe special party-sceneid verified accounts)... then include the need of an unpacked exe in the party-rules for compo entering (this could be a well-defined zip format or something where an automatic tool could be made).

This would work for party release distribution only though, but I guess that'd cover most of the needs for such a tool (and special requests could be made to scene.org to sign off-party executables, there shouldn't be that many).

Also, if party organizers and scene.org agree on the terms, the cost of the certificate (and maybe the maintaining of the service) could spread among all subscribing demoparties, which would make it quite cheap.
added on the 2015-04-23 20:17:17 by Jcl Jcl
This idea is going to be really popular:
How about making it mandatory to release source, assets and build instructions along with the executables?
Yeah, I know I'm kidding myself.
added on the 2015-04-23 23:34:25 by cxnull cxnull

And if you check the comments, some newfaggot flagged conspiracy.hu as possible malicious domain because some snake oil company blacklisted them for no reason.
"ORDER WEB SITE CLEANUP AND BLACKLISTING REMOVAL!" BB Image
added on the 2015-04-24 10:58:41 by merkur merkur
Quote:
How about making it mandatory to release source, assets and build instructions along with the executables?


Although, I'd like to see the inner workings of several demos, the amount of effects, code, textures and music rip offs would increase dramatically.
added on the 2015-04-24 11:24:25 by Defiance Defiance
You shouldn't have to release your entire source to prove something is not a virus. Period.

My limited knowledge of security suggests that credibility rather than transparency can accomplish the same goal.

Could kkrunchy be modified to integrate some kind of PGP signing?

Could a PGP solution work? Or will this not satisfy crappy antivirus software?

At the very least, bothering to associate a key with each release would show folks are trying to make the software trustable and traceable, which would increase credibility and make folks less suspicious of packed releases.

I believe that various group software projects have also encountered some of these issues, in a different context of course . . . .

See:
https://www.apache.org/dev/release-signing.html#motivation
metoikos: That is essentially what BoyC explained he did in the first post, except that it's not PGP but rather Windows' integrated code signing.

login