pouët.net

packers/compressors & security & malware & webhosting

category: code [glöplog]
I understand that we have two different problems.

One is the webhosting of files that could be marked as viruses by AV software.

The other is the AV software that mark our intros as malware.

In fact, the real problem is the second one because most people confide in AV software and, while most of AVS continue to delete our stuff, very few people will jeopardize their computers executing that "demoscene things".

We can, for the moment, solve the webhosting problem in many ways. We could use untergrund and scene.org as safe webhosting or try what Tigrou & Psycho said if our webmasters are inflexible and obstinate robots who don't want suspicious files stored.

But, if we achieve to stop the detection by the stubborn webmasters, that doesn't prevent the AVS installed in the visitor's computer to raise the "warning... this file is surely a virus and so I, your kind AV, delete it now for your own security before you execute it!" alert that will scary lots of visitors and, probably, in the end, blacklist the page as potential malvare site sooner or later.

So... probably the best solution, as some people in this thread said, is to distribute the intros as data (xorted binaries, for instance) and release a tool that convert the datafile in an executable intro. This way, the user of the tool take the risk of create an executable file that his/her AVS will surely delete or put in quarantine and he/she do it voluntarily after reading a text explaining what is an intro and why some AVS could do a false positive.

I agree with speeder that we should not made packers that try to hide their signatures as they should be used soon by malware creators as great tools to make invisible viruses.

I like the idea of xernobyl too but I am not sure if that even could be possible. Malware friend functions? Well, maybe ryg, or other executable packer author, could prevent the use of a few functions but I find hard to believe that AV companies would be convinced that every file packed with a magic "safecruncher" should be in their whitelist.

Still I think that this security paranoia will not end easily.
added on the 2010-06-12 16:11:35 by ham ham
When it comes to webspace and distribution of intros a screenshot is enough to add a 4k or even a 64k zip at the end. Somebody have to tell me if this works on Vista or Seven cause it still does on XP using WinRAR. I'm not sure if webhosters do check jpg files too cause they more or less picture data.

Its quite a strange thing.
added on the 2010-06-12 17:10:36 by yumeji yumeji
i think AVS (and most antivirus scanners) works this way :

1. check the header of the file (and not the extension) to know what it is :

its a file that contains executable code (thus harmfull)
=> scan it (step 2)

otherwise it should be data file
=> dont do anything this file is virus free or (do a full bytecode pattern search to be sure...)

2. check the structure of that file in the case it is an executable

its a well know structure
=> read it and go even deeper (step3)

the structure is not know or is to complex to be fully checked
=> we dont know what it is but this is marked as executable
this file is suspiscious and considered as harmfull (since we cannot decide if its clean or not). this is the case of iq intros where the packer is too complicated to be analysed...

3.

check the imported functions and everything that looks strange

4. search in the executable virus bytecode patterns an so....
added on the 2010-06-12 17:29:21 by Tigrou Tigrou
I encountered an antivirus (Norman, I think) which tries to be smart about unknown executables. When it encounters an unknown exe file, it will run it inside a sandbox virtual machine and monitor its behavior to check that it does not do anything suspicious.

In order to avoid heavy load on the system during these executions (which could be part of a background scan), it restricts the memory allocated inside this virtual machine to 64MB. And that is where it all falls down.

Crinkler-compressed executables start out by doing a lot of random accesses in a hashtable of at least a hundred megabytes as part of the decompression process. When run with only 64MB available, this process will take hours at best. The result: The AV scanner scans happily along, hits a Crinkler-compressed file and stops dead in its tracks...
added on the 2010-06-12 23:21:05 by Blueberry Blueberry
if the problem is doing proper unpack tests then why the hell wont av's just send the file to a central server that checks against a whitelist or performs proper indepth scan before going all "ALERT ALERT ZE GERMANS ARE COMING" on us? just checksum the damn few suspicious files and send them to a centralized (or even p2p) whitelist system of sorts. learn something from the anti spam systems..

why arent there digitally signed whitelists of potentially dubious files? AVs should be battling for who has less false positives benchmarking those repositories.

i would find acceptable as a developer to submit my packed file once to a trustworthy AV to have it whitelisted everywhere if that would allow any and all potential users the possibility of not being falsely nagged about my exe if they choose to use an AV.
added on the 2010-06-13 02:12:23 by psenough psenough
hell, we could even host one at scene.org for all demoscene exe files for free if AVs would support the damn thing. who do we need to spam-nag to death to get this BS sorted?
added on the 2010-06-13 02:14:03 by psenough psenough
Quote:
Crinkler-compressed executables start out by doing a lot of random accesses in a hashtable of at least a hundred megabytes as part of the decompression process.

Ouch. Good thing the kkrunchy context model is comperatively tiny at about 6MB. :)
added on the 2010-06-13 04:04:56 by ryg ryg
Fucking end the topic !! ..

Just deliver a AV compatible unpacked 64k or 4k !! .. Even if the effects are less than what you guys wanna do !! .. It's still the same artist thing involved !! .. It's just the uncompressed size that matters .. with that !! ..

And you all know howto make it smaller .. even uncompressed !! .. The worst you might do is a oversized demo that didn't compile and compress into a 64k or 4k !! ..

Just fuck the AV scanners !! .. and do it the usual way !! .. The scene is barely alive .. anyway !! ..

Just do it for the cause !! .. the art !! .. :) ..

Have you ever thought about that ??
added on the 2010-06-13 06:17:23 by yumeji yumeji
why don't u share decompressor code with AV developers? it's not a commercial software and i don't really see any commercial application to it. except for virus makers but fuck them...

AV (many of them) can depack UPX-packed files and check them. why don't you guys do the same for kkrunchy and crinkler? they might be interested in such compatibility... after all, it will help to get rid of a lot false-positive results in their scanners
added on the 2010-06-13 07:49:49 by RRROAR RRROAR
Quote:
AV (many of them) can depack UPX-packed files and check them. why don't you guys do the same for kkrunchy and crinkler?

Right, why bother reading any of the thread before you post?

Quote:
they might be interested in such compatibility... after all, it will help to get rid of a lot false-positive results in their scanners

Since the false positives in a few hundred files that none but a few hundred customers are even interested in hurt them so badly? Yeah right. The whole reason this problem exists is because there is no economic case for AV vendors to spend any time on reducing false positives on what is to them obscure software.
added on the 2010-06-13 09:19:04 by ryg ryg
We need to infiltrate the AV industry!
added on the 2010-06-13 09:59:10 by msqrt msqrt
wow, this is really annoying what did your web hosting provider iq... Concerning this, I would switch to a web hosting that doesn't scan your stuf... even if the php decrypt trick is easy to do, I wouldn't bother with it if possible...

Concerning dekstop AV, well, there is probably no easy solution... it would require a proprietary compressor not released to the public. A kind of "demoscene compressor web-service" using a non-public compressor could be created, with trusted approver... but that would require for the moderator a manual check of the intro.exe to insure its safety and its demo commitment... almost impractical...

Quote:
Ouch. Good thing the kkrunchy context model is comperatively tiny at about 6MB

wow... 6Mb for the probability counters? With this size, I suspect you don't really use any hashmap... are you recalculating counters from the beginning for each model and new bit to evaluate or something else?
added on the 2010-06-13 10:31:09 by xoofx xoofx
Even though it's their software that accuses legitimate programmers of writing malware, it does so systematically and in a deliberately predefined way, leading many to falsely think intro coders are malware writers. It's not as if this is some malfunction of the AV software. It's how it's intended to work. And it's not like they're not aware of the false positives, or haven't been offered very good ways of solving the problem. They still insist on labeling a whole category of legitimate software as malware. So surely they're liable for slander, right?
added on the 2010-06-13 10:31:49 by doomdoom doomdoom
weird, trying Jotti online multiple AV virus checker on some crinklerized 4k exe, only 3 other 19 are giving false alarm... tested for example on Tbc-Receptor results here... don't know if this service comparison is reliable, but if yes, should we bother with desktop AV?
added on the 2010-06-13 10:49:49 by xoofx xoofx
Have you try with virustotal.com ? i try with tons of antivirus software soft would probably give more accurate results.

Here is a list :
Quote:
AhnLab (V3)
Antiy Labs (Antiy-AVL)
Aladdin (eSafe)
ALWIL (Avast! Antivirus)
Authentium (Command Antivirus)
AVG Technologies (AVG)
Avira (AntiVir)
Cat Computer Services (Quick Heal)
ClamAV (ClamAV)
Comodo (Comodo)
CA Inc. (Vet)
Doctor Web, Ltd. (DrWeb)
Emsi Software GmbH (a-squared)
Eset Software (ESET NOD32)
Fortinet (Fortinet)
FRISK Software (F-Prot)
F-Secure (F-Secure)
G DATA Software (GData)
Hacksoft (The Hacker)
Hauri (ViRobot)
Ikarus Software (Ikarus)
INCA Internet (nProtect)
K7 Computing (K7AntiVirus)
Kaspersky Lab (AVP)
McAfee (VirusScan)
Microsoft (Malware Protection)
Norman (Norman Antivirus)
Panda Security (Panda Platinum)
PC Tools (PCTools)
Prevx (Prevx1)
Rising Antivirus (Rising)
Secure Computing (SecureWeb)
BitDefender GmbH (BitDefender)
Sophos (SAV)
Sunbelt Software (Antivirus)
Symantec (Norton Antivirus)
VirusBlokAda (VBA32)
Trend Micro (TrendMicro)
VirusBuster (VirusBuster)
added on the 2010-06-13 11:22:01 by Tigrou Tigrou
msqrt: there are some sceners working on the AV industry. mostly at Panda. not sure how much they can push to fix this issue though. changing companies policies is always a nightmare. especially when the market isnt demanding it now.
added on the 2010-06-13 16:00:20 by psenough psenough
Indeed, the VirusTotal is really nice...

Nice is not what happened with my game :(

I sent my game .exe packed with MEW11...

11 positives (among 41 AV software):

Authentium 5.2.0.5 2010.06.13 W32/Heuristic-210!Eldorado
AVG 9.0.0.787 2010.06.13 Suspicion: unknown virus
Comodo 5088 2010.06.13 Backdoor.Win32.IRCBot.~HYJ
eSafe 7.0.17.0 2010.06.13 Win32.Stration
F-Prot 4.6.0.103 2010.06.13 W32/Heuristic-210!Eldorado
McAfee-GW-Edition 2010.1 2010.06.12 Heuristic.LooksLike.Win32.Suspicious.R
Sophos 4.54.0 2010.06.13 Mal/EncPk-BU
TheHacker 6.5.2.0.298 2010.06.12 W32/Behav-Heuristic-066
TrendMicro 9.120.0.1004 2010.06.13 Cryp_MEW-11
TrendMicro-HouseCall 9.120.0.1004 2010.06.13 Cryp_MEW-11
VirusBuster 5.0.27.0 2010.06.13 Packed/MEW

Later I will send a UPX version and see what happens...

I dunno what is more hilarious: "Backdoor IRC" or AVG flagging it is "unkinown virus"
added on the 2010-06-13 20:35:43 by speeder speeder
I sent the same software, using UPX "brute mode"

For some reason, only 37 AV software tested it (Oo).

But none returned a positive :) Yay! UPX to go!

Also UPX in "brute mode" compressed better than MEW11 (although, the file was originally 6mb big, and UPX shaved 3kb more...)
added on the 2010-06-13 20:53:19 by speeder speeder
Let´s face it: AV and "security" software has reached a point where it does more damage (including broken backup scripts or even operating systems rendered unuseable).

And I suspect that the vendors are pretty happy with their software actually beeing scareware and don´t care about false positives at all: If some less commonused tools and websites generate "suspicious virus/attack/malware" alarms it makes their software look useful - the average user won´t even notice that and remains fooled successfully, thus getting the idea that the scareware actually seems to be useful.

C´mon, look at the signatures - they actually tell that they have detected a packer or joke tool, yet they still mark them as "virus", "trojan" or "malware" without actually telling the user the truth which would be no worse than "suspicious, might be harmful or not".

I also got an UPX compressed file marked as "Troja / X Pack" or sth. like that by several vendors, thus even a rather notable packer like UPX won´t safe it.

I´d be rather willing to sue the AV vendors for their hillarious disinformation and potentially repuration-wreckig behaiour, but given the fact that
-they often operate from abroad
-there´s no such thing like a class-action lawsuit here
-software vendors are likely to have pretty skilled lawyers
-the chance of getting a judge who is to dumb to understand what´s really going on is pretty high
makes it unlikely to succeed, thus ending up even worse than before
added on the 2010-06-14 03:11:08 by T$ T$
Quote:
Fucking end the topic !! ..

Just deliver a AV compatible unpacked 64k or 4k !! .. Even if the effects are less than what you guys wanna do !! .. It's still the same artist thing involved !! .. It's just the uncompressed size that matters .. with that !! ..

And you all know howto make it smaller .. even uncompressed !! .. The worst you might do is a oversized demo that didn't compile and compress into a 64k or 4k !! ..

Just fuck the AV scanners !! .. and do it the usual way !! .. The scene is barely alive .. anyway !! ..

Just do it for the cause !! .. the art !! .. :) ..

Have you ever thought about that ??


sounds like you were new to the demoscene
added on the 2010-06-14 06:15:55 by iq iq
I registered to untergrund a couple of days ago. Any idea how much should I expect to wait for an answer? Just so I know.
added on the 2010-06-14 06:17:01 by iq iq
When I'm drunk i write the craziest bullshit :D

Quote:
sounds like you were new to the demoscene


Sober now.

Don't call me new. Already 10y watching all the stuff and coding bullshit myself sometimes. Just to slow or bad coding something worth it to be released. I moved from my bored 64k to a 4k so I know why I need the compressipon myself. But I hate the fuckery with the stupid AV corps here in the thread. Just give them a shot and done. They'll never safely whitelist exe-packers cause maybe they're used to compress malware and the mechanism can't deal with the compressed code. Just work around or don't use or ignore them.

I don't really care. So you know.
added on the 2010-06-14 09:56:09 by yumeji yumeji
Attention all: Yumeji doesn't care!
added on the 2010-06-14 10:46:43 by sagacity sagacity
Oh shit, he doesn't? What the fuck do we do now??
added on the 2010-06-14 11:02:33 by okkie okkie
TBH I was assuming you had a plan.
added on the 2010-06-14 11:23:27 by sagacity sagacity

login