Digitally signing kkrunchy compressed releases
category: code [glöplog]
After this Revision we got harshly reminded of an old problem: 33 out of 57 antivirus packages on virustotal reported our intro as malware. When we last dealt with this a couple years ago, one response to our whitelist requests was that we'd be better off signing our executables with a trusted certificate.
I tried looking into this last year after nvscene and immediately ran into issues. The problem was that kkrunchied programs don't register as executables for the microsoft signing tool. Well, today after a bit of reverse engineering windows dlls I found the cause and managed to create a kkrunchy version that allows the digital signing of releases, at the cost of about 1 kbyte. I think this cost is very reasonable if it can get rid of all the antivirus false-positive crap. At the very least it allows for the possibility of distributing signed, "safe" versions of releases.
There is, however, the issue of the certificate. I'm not too well versed in this area, but as this issue affects several groups I'm wondering if there would be interest in pooling our resources into a common Demoscene certificate that might be used for signing releases. As this kind of thing would certainly make outreach easier, maybe even some of the organizations that are working towards that goal could help out. I know there are problems with the idea (for example security of a shared certificate is almost guaranteed to go wrong), but I'd like to hear your thoughts.
I tried looking into this last year after nvscene and immediately ran into issues. The problem was that kkrunchied programs don't register as executables for the microsoft signing tool. Well, today after a bit of reverse engineering windows dlls I found the cause and managed to create a kkrunchy version that allows the digital signing of releases, at the cost of about 1 kbyte. I think this cost is very reasonable if it can get rid of all the antivirus false-positive crap. At the very least it allows for the possibility of distributing signed, "safe" versions of releases.
There is, however, the issue of the certificate. I'm not too well versed in this area, but as this issue affects several groups I'm wondering if there would be interest in pooling our resources into a common Demoscene certificate that might be used for signing releases. As this kind of thing would certainly make outreach easier, maybe even some of the organizations that are working towards that goal could help out. I know there are problems with the idea (for example security of a shared certificate is almost guaranteed to go wrong), but I'd like to hear your thoughts.
A demoscene-wide certificate is likely to get abused/invalidated/ignored soon, thus a certificate for a single group (or an association of few groups) is likely to work better.
1k for 64k is usually bearable (an executable icon is similar in size footprint and quite common), so it might be a good option. Too bad this doesn´t apply to 1k/4k/8k.
Personally, I´d rather prefer using up the remaining bytes rather than adding a certificate forav gone wrong, but often there are some bytes left anyway so that´s better than zero-padding the exe or adding tons of generic scrolltext.
1k for 64k is usually bearable (an executable icon is similar in size footprint and quite common), so it might be a good option. Too bad this doesn´t apply to 1k/4k/8k.
Personally, I´d rather prefer using up the remaining bytes rather than adding a certificate forav gone wrong, but often there are some bytes left anyway so that´s better than zero-padding the exe or adding tons of generic scrolltext.
Great initiative! I like it (especially that you've already have some working solution!)
Quote:
A demoscene-wide certificate is likely to get abused/invalidated/ignored soon, thus a certificate for a single group (or an association of few groups) is likely to work better.
It could work if the certificate is not spread around, i.e. intro makers would have to send their certificate to scene.org and get it signed there. Of course that has the disadvantage that someone outside your group is going to see the release early. :)
First question would be the legality of actually sharing a cert. (Pretty sure CAs have wording forbidding that, or at least making it difficult.)
The only way I can see to share such a cert somewhat securely would be to have a trusted central entity that would receive all executables to sign, and then sign them (possibly even through some form of semi-automated process where you upload your release, somebody clicks the approve button and out comes your signed executable).
That still leaves the question for who pays the cert, and how does one determine whether to approve an executable or not (proven scene.org ID with affiliation to known group? ...).
The only way I can see to share such a cert somewhat securely would be to have a trusted central entity that would receive all executables to sign, and then sign them (possibly even through some form of semi-automated process where you upload your release, somebody clicks the approve button and out comes your signed executable).
That still leaves the question for who pays the cert, and how does one determine whether to approve an executable or not (proven scene.org ID with affiliation to known group? ...).
sounds like a nice feature for scene.org or demozoo or those german demoscene outreach associations.
"Sincerely yours,
Demoscene"
Demoscene"
How would scene.org make sure they are not signing a virus?
Sesse, they wouldn't. Unless someone took the time to check each one thoroughly before approving it to be signed. BoyC alluded to this in his first post:
Quote:
by BoyC:
I know there are problems with the idea (for example security of a shared certificate is almost guaranteed to go wrong), but I'd like to hear your thoughts.
Yeah, automated signing is problematic.
Maybe it could be made available at parties for entered releases, or after a party for the prods (this is not optimal of course as unsigned versions would also be in circulation)? Some obvious limitations can also be placed on the exe being signed: has to be below 64k for intros, has to be compressed with kkrunchy, etc.
Maybe it could be made available at parties for entered releases, or after a party for the prods (this is not optimal of course as unsigned versions would also be in circulation)? Some obvious limitations can also be placed on the exe being signed: has to be below 64k for intros, has to be compressed with kkrunchy, etc.
Make scene.org do not only the signing, but also the build from source. But that would be a hopeless amount of work given the average demoscener's build setup :-)
What's the benefit of a shared certificate? Saving money?
For one, yeah.
I also have the suspicion that if we start signing compressed executables we'll still need to go through the same hassle of whitelisting, only it'll be enough to do it once for the cert.
I also have the suspicion that if we start signing compressed executables we'll still need to go through the same hassle of whitelisting, only it'll be enough to do it once for the cert.
Just give all non-sceners the youtube link. No need for .exe distribution.
Experimenting with this it seems that if I add timestamp the exe as well this starts getting prohibitively expensive (~5.5k), and looks like if the cert is the result of a chain of CAs it'll be even larger.
At any rate I'd still prefer to have the option of distributing an executable that doesn't set off alarm bells everywhere, even if it's just in the form of providing a "safe download" on our website that exceeds the original size limitation by a few kbytes. For such purposes the ability for scene.org to sign releases after the party would be a welcome addition.
At any rate I'd still prefer to have the option of distributing an executable that doesn't set off alarm bells everywhere, even if it's just in the form of providing a "safe download" on our website that exceeds the original size limitation by a few kbytes. For such purposes the ability for scene.org to sign releases after the party would be a welcome addition.
imho "fixing" programs just so broken snakeoil heuristics dont complain is just terribly wrong and counterproductive. i'd rather see you putting that effort into filling the remaining 1k with more awesomeness.
groepaz: the problem is that i've spent most of my free time this week sending whitelist requests, some of which only have a temporary effect (sophos for example whitelisted the intro for a couple hours today but it detects it again now...), not to mention the time a few years back when our complete website was blocked from some locations due to some av software going nuts. If it means that we don't have a fancy loading bar and an icon in our intros so be it. If we're short on space for a particular release we can always decide to drop these 'ballast' features, but to at least have the option of getting around all this av crap would be nice.
(Also, i found a 'better' timestamping server, the signature is about 2.7k this way, within acceptable range imo. I asked around on the Comodo forums if maybe someone is willing to make a test signing with a proper certificate, hopefully I'll get some real-life info that way without having to buy a cert :)
(Also, i found a 'better' timestamping server, the signature is about 2.7k this way, within acceptable range imo. I asked around on the Comodo forums if maybe someone is willing to make a test signing with a proper certificate, hopefully I'll get some real-life info that way without having to buy a cert :)
Any chance of source code for these modifications? Do you still use MS's signtool, or does kkrunchy sign the exe itself?
mudlord: the root of the problem was that the signtool doesn't recognize a binary as an executable if the offset to the PE header located at 0x3C points to a value lower than 0x3C, everything else checked out fine.
So the fix is about 3 lines:
in the exepacker.cpp replace this
to something like this
and remove the now not needed line
And yeah, I use signtool.exe for testing. I figured it's a better option to stick to an "official" solution instead of forcing one because the PE checking code is located in wintrust.dll, and I suspect it's also used when a certificate is being validated.
So the fix is about 3 lines:
in the exepacker.cpp replace this
Code:
AppendImage("MZfarbrausch",12); // dos stub
OPH = (PEHeader *) AppendImage(PH,sizeof(PEHeader));
to something like this
Code:
AppendImage("MZfarbrausch ist gut farbrausch ist scho:n farbrausch ist toll",0x3c);
int pos = 0x40;
AppendImage(&pos, 4);
OPH = (PEHeader *)AppendImage(PH, sizeof(PEHeader));
and remove the now not needed line
Code:
OPH->DataStart = 12;And yeah, I use signtool.exe for testing. I figured it's a better option to stick to an "official" solution instead of forcing one because the PE checking code is located in wintrust.dll, and I suspect it's also used when a certificate is being validated.
If you upload intros to virustotal.com you shouldn't wonder that stuff gets detected quite fast.
Nevertheless, I don't think we should do that certificate stuff, it's already broken.
False positives happen all the time (not only with virus scanners) and if people fail to understand that - it's not your fault and you shouldn't care about it too much (sometimes not too easy, I know).
=> Maybe this should be a bit more about education than about buying certificates.
Nevertheless, I don't think we should do that certificate stuff, it's already broken.
False positives happen all the time (not only with virus scanners) and if people fail to understand that - it's not your fault and you shouldn't care about it too much (sometimes not too easy, I know).
=> Maybe this should be a bit more about education than about buying certificates.
las: Can non-coders/non-geeks, like "normal" people interested in realtime art, really be expected to understand this issue, let alone be burdened to judge if a given exe is really a false positive and not malware? That's asking a bit much IMHO.
BoyC: be aware that virustotal shares all findings with av vendors. So if your intro gets some false positives during testing there, chances are high that all other scanners will "detect" it as malware afterwards as well.
But if it's only so you can provide an exe outside of parties for people to be able to watch it "safely": Why not offer an unpacked version for download as well?
BoyC: be aware that virustotal shares all findings with av vendors. So if your intro gets some false positives during testing there, chances are high that all other scanners will "detect" it as malware afterwards as well.
But if it's only so you can provide an exe outside of parties for people to be able to watch it "safely": Why not offer an unpacked version for download as well?
People that use AVs deserve to receive fake alarms all the time, because they're THAT stupid. The more alarms, the faster people will uninstall their AV and return to sanity.
... one can dream, right?
But yes, if you're going to end up cramming a cert in there, go for a "safe" version that is a bit aboce the size limit. There's simply no reason to lose precious bytes just because of a stupid cert.
... one can dream, right?
But yes, if you're going to end up cramming a cert in there, go for a "safe" version that is a bit aboce the size limit. There's simply no reason to lose precious bytes just because of a stupid cert.
fgenesis, that's a stupid statement. AV software is of course not perfect, especially against new malware for a couple of hours until signatures have been created and deployed. But it's still a necessity to protect against all the other malware crap that's around (Conficker from 2008 is *still* in the top 10 of infected machines worldwide...).
I wish everyone who says people should uninstall av software their own ransomware incident so they shut up...
Oh, and as annoying as it is, I can see why av vendors flag intros as malware. Outside the demoscene, there's not much reason to have an exe-packed, "obfuscated" (i.e. size-optimized) program which is not signed or white-listed. This heuristic shouldn't produce much false positives outside of our niche.
I wish everyone who says people should uninstall av software their own ransomware incident so they shut up...
Oh, and as annoying as it is, I can see why av vendors flag intros as malware. Outside the demoscene, there's not much reason to have an exe-packed, "obfuscated" (i.e. size-optimized) program which is not signed or white-listed. This heuristic shouldn't produce much false positives outside of our niche.
include the unpacked exe too.
teo: that would still flag the .zip as malicious by some very proactive scanners, as the virus is in the .zip
