You mean the tons of md5 windows hashers? Like hashtab, md5summer, sfv32 and the rest? Or about the included ones?
The fact that you are not familiar with them doesn't mean no one else is.

WTF, this check should be the default for downloads!
Allowing an undetected chance of file corruption is absurd..

I wonder.. is this the same when you are moving files on other hdisks? O_o
(I'm using Teracopy for that)

Corruption is always possible, though usually the fault of memory. There are various integrity checks, but they do take time. So unless you have reason to believe there is a chance at corruption, you can ignore them.

Edit: Teracopy does have an integrity check from what I recall, but not sure if it's optional or not.

I remember reading an article and it said something like there were a million corruptions and tiny errors on typical hard drives (Don't ask for a source as i don't remember where i read or saw it at); But many of those are low level and error correction and detection usually fixes them. Computerphile went into detail on this as well, showing that a music CD could still be played even after drilling a hole into the CD due to the error correction built into the format of the media.

Back in the 60's and 70's it was a mystery as to why some communications and file transfers were fine, and others failed (something like one in a hundred). There was some low fractal noise that was present (due to the electricity or something?) That couldn't be removed. This meant the easiest solution was to add checks, and if the check failed you simply redownload that portion that failed. TCP/IP integrates CRC checks, but sometimes you still get a bad batch when the entire download is compared vs the individual ones that all passed.

Corruption in text files are unnoticed as there's enough redundant data in language you mentally correct it as you read. Pictures may have a blip of odd data, or if it's bad, it will bleed odd colors and repeats of previous portions of the image over and over again. Audio will likely only have a blip of sound, or if it detects the error it will skip past the error and keep playing at a point it can confirm is still good (same for video). For lossy media that isn't so bad.

For programs and things that MUST be lossless, this is where the failures really show and why the data checks are so important. As mentioned, few programs actually check it during download, but you can manually confirm it using an archiver (if the archive format supports it. Most do though).


Actually sometimes CRC checks are too prevalent. Many PS1 games have lots of extra error checking and redundancy. There's actually a set of programs to remove the error checking if it conforms to a specific format from the ISO's of PS1 for space, then the other program puts it back in. (Since the compression and archives contains it's own CRC and error checking, duplication often is unneeded) which drops the hugs ISO's down something like 50Megs per disc.

+1 for the detailed explanation.
An option like "always verify data for these file types" (.exe,.zip,etc..) would be good for downloads, however..

Agreed. I see quite a few installers (Like GoG) take a while to start up while it tests itself; Archives will simply fail and tell you if it encounters errors, which is horrible if you no longer have internet access at that point. I've seen notes regarding zlib and gzip where during testing they intentionally randomly enter errors and then the decompressor will fix them, but it takes time as it has to figure out where the error is at and then brute force changes until it comes out correctly. I think the testing was with 1-2 bits wrong, although it could have been bytes.

So even with corrupt data all is not lost. But it's not good either...

edit: I forgot to mention low level error checking that goes into basic communication. Serial lines (5 pin) where individual bits are transmitted. Usually a start bit and stop bit, and error test bit of even/odd. This usually increases 8 bits of information to 11 bits for basic error detection.