Have you even been in a situation that you discover your hard drive from 10 years ago and you say to your self, “I have some good things saved here, lets see them”, only to discover that HDD may start to fail after years of sitting forgotten in a box hidden in a dark corner in you house?
Well, I have an experience like that the last couple of days.
I have made a clean-up to my old boxes, and i have made new once! I know … I know… It’s not the best de-cluttering solution, But at least my cable-box is smaller, and my hardware box is more tidy! Along with that, I also tested my old hard-drives. Mainly Laptop HDD’s 2.5” 500GB-1TB, external HDD’s 2.5” 1TB or 2TB and some legacy 3.5” HDD 250GB!
All of my Laptop hard drives where Good as new. Along with my 3.5” 250GB that i don’t even remember from what era i had saved it. ( I don’t but the files there are from 2008! ).
I have started migrating any file worth of saving to my Desktop HDD’s that under raid, only to find out that one of my external 1TB hard drives (Samsung m3 portable, that i think i have it since 2012 or 2014) is having a lot of CRC errors. For you that have just READ CRC and you don’t know what it is, Let me tell you.
What is a CRC error?
Cyclic Redundancy Check (CRC) Error indicates when data is corrupted. Calculating from all data, CRC validates packets of information sent by devices and verifies it against the data extracted, ensuring its accuracy. When sending packets over,BACnet automatically calculates and stores a CRC value for the packet. Later when the packet is received, BACnet calculates the CRC value of the packet and compares it to the original value. If the two values do not exactly match a CRC error occurs.
source: https://www.setra.com/blog/what-is-crc-error
I have CRC error… what do i do now?
If you find yourself in that situation, then you will probably try and retry to copy the files to a new hard drive! It’s perfectly reasonable to get obsessed about it, BUT this will push your hard drive to its limits, making it to fail faster.
Well.. i would suggest to start by either copying the rest of the files, saving what you can. STARTING from the most important files, as you don’t know when your hard drive is going to fail. For me, i saved about 450GB from my 935GB that i have in that HDD. If you ask me why do you want to save 935GB? the answer is simple. This HDD has a lot of files from my college and masters years, along with many e-books, DnD (Dungeons and Dragons) data, and Music studies files. That’s why i want to save the files. The bad thing is that my HDD got corrupted in such a way, that it wasn’t even possible to mount it. So here are my next steps that i have tried in order to save my files.
Steps of salvation
Step 1: chkdsk
Since i was working with an HDD that was ntfs, i tried to repair it using chkdk I: /r /x
initially i got good results, that made my disk a little bit stable, and i do recommend that you try it as well.
Unfortunately for me this disk was in such a bad shape, that my windows computer starting to mount and unmount it in random intervals, and the chkdsk command could not finish.
Step 2: MacOS?
I tried mounting the disk to my macbook laptop, but the macos, since it does not natively support ntfs filesystem, (Mine does as i have extensions), couldn’t even mount my disk… so no chance there…
At least this step made sure that the problem was with the disk and not with my Desktop.
Step 3: Check disk controller.
The point with this step, especially with external HDD’s is to check that the SATA to USB controller is in good shape. In order to check that i had to open the external samsung M3 portable case, and disconnect the HDD from the controller. WELL SAMSUNG M3 doesn’t have a controller! the USB port is soldiered to the HDD! (i will never get an other SAMSUNG external hard drive!)
Step 4: Try to mount the Disk to my Linux computer!
Again My Disk is unmountable! The is no way i would ever see my files again! OR IS THERE ?
Step 5: Recovery!
There are numerous ways to recover your Hard Drive. There are a lot of programs that with the right amount of $$ or €€ or ££ or ¥¥ will do it for you! (If the disk is not getting unmounted in your windows machine). But that’s not my preferred method.
My preferred method is a linux program called ddrescue .
ddrescue , is a specialized dd program, that what it does, is to make an exact byte-to-byte copy of a folder, file or HDD! Basically it uses the old CD/DVD copy methodology. which is: Copy what ever you can see, DO NOT READ IT, just make a copy of the sequential 01010101010011 patterns.
OK, it’s not that simple, that is just the theroy, ddrescue is smart. it read, tries to comprehend, and if it cannot, then it just makes the copy. Basically this program tries to repair the data if it’s possible.
How to install ddrescue
I will give you instruction about ubuntu installs.
In ubuntu Linux you first need to enable universe repositories:
sudo add-apt-repository universeThen just update your repos and install it
sudo apt update
sudo apt install gddrescueTo run ddrescue, use the following format for commands:
ddrescue [options] infile outfile [logfile]ddrescue Command Explained
Here’s an explanation for some of ddrescue options:
-fForce ddrescue to run even if the destination file already exists (this is required when writing to a disk). It will overwrite.-nShort for’–no-scrape’. This option prevents ddrescue from running through the scraping phase, essentially preventing the utility from spending too much time attempting to recreate heavily damaged areas of a file./dev/[baddrive]Identifies the source drive that will be copied. Fill this in with the name of your bad drive./dev/[gooddrive]or/root/[imagefilename].imgIdentifies the destination drive or image file where the data will be cloned. Fill this in with either the name of the good drive or the image file name of your choice./root/recovery.logThis creates a logfile, which is essential if you’re performing multiple passes. You can name the logfile anything. Without a logfile, you can’t make additional passes on areas of your disk with bad sectors.
- Short for ‘–sparse.’ This compels ddrescue to use sparse writes — blocks of zeroes aren’t allocated on the disk, which can save space. However, it can only be used for regular files, and it is not an available option on all operating systems.
-r3Tells ddrescue to keep retrying damaged areas until 3 passes have been completed. If you set ‘r=-1’, the utility will make infinite attempts. However, this can be destructive, and ddrescue will rarely restore anything new after three complete passes.-DShort for ‘–synchronous’. This issues an fsync call after every write.-dShort for ‘–delete-if-done’. Deletes the given logfile “if all the blocks in the rescue domain have been successfully recovered.”-e [+]nShort for ‘–max-errors=[+]n’. This sets the maximum number of error areas allowed before ddrescue gives up, and it can be used to prevent the utility from running infinitely.-vShort for ‘–verbose’. This sets “verbose” mode, providing additional details. Can be useful for diagnosing issues.-SShort for ‘–sparse.’ This compels ddrescue to use sparse writes — blocks of zeroes aren’t allocated on the disk, which can save space. However, it can only be used for regular files, and it is not an available option on all operating systems.
A full list of usable commands can be found here in the ddrescue manual.
Precautions When Cloning with ddrescue
ddrescue is a powerful utility and should only be utilized by experienced Linux users. It can cause damage to a failing hard drive in some circumstances. It can also overwrite data when used incorrectly.
You should immediately turn off your computer if the failing hard drive seems to get worse, for instance if sounds get louder or if successive passes result in a larger number of bad sectors.
Be sure that the good drive doesn’t have any important data that might be overwritten. Make sure that you properly identify the good and bad drives when using the utility. Check for capitalization errors before starting the process, and don’t leave the hard drives unattended (if you do, you won’t be able to turn off the computer if serious errors occur).
What exactly did i use to make the backup?
sudo ddrescue -r3 -f /dev/sdg /dev/sdh clone.logfileI pulled one of my Old Laptop 2.5” HDD 1TB that i had just migrated to my Raid HDD’s in my Desktop, and i plugged it to an external HDD usb dock.
/dev/sdg if my Failing corrupted HDD
/dev/sdh is the disk i am going to save the data
clone.logfile is the logfile I instruct ddrescue to create, in order to have a log if i need to restart the proccess for what ever reason. ddrescue takes A LOT of time to complete, especially on failing hard drives. In the screenshot it says remaining time: 14d 3h 14m. This is not actual, it fluctuates, as ddrescue and all recovery software take a LOT of time to read over bad sectors or bad blocks of data. sometime is says 5hours sometimes 112DAYS !
At the time of writing, I am still waiting for the recovery to finish.I will get back to you as soon … as it’s done.Stay tuned for the updates!
[~2Days Later] While we wait [Update]
Lets read some basics about the ddrescue algorithm as written Here
Algorithm
Only non-tried areas are read in large blocks at Copying phase. Trimming, scraping, and retrying are done sector by sector.
Each sector is tried at most two times; the first in the Copying phase as part of a large block read, the second in one of the phases below as a single sector read.
1. Copying phase
Copying is done in up to five passes. The copying direction is reversed after each pass until all the rescue domain is tried.
The purpose of the multiple passes is to delimit large bad areas fast, recover the most promising areas first, keep the mapfile small, and produce good starting points for trimming.
- First pass: The first pass reads the non-tried parts of the input file, marking the failed blocks as non-trimmed and skipping beyond them.
- Second pass: The second pass delimits the blocks skipped by the first pass. The first two passes also skip beyond slow areas.
- Third and fourth passes: The third and fourth passes read the blocks skipped due to slow areas (if any) by the first two passes, in the same direction that each block was skipped. For each block, passes 2 to 4 skip the rest of the block after finding the first error in the block.
- Fifth pass: The last pass is a sweeping pass, with skipping disabled.
2. Trimming phase
Trimming is done in one pass. For each non-trimmed block, read forwards one sector at a time from the leading edge of the block until a bad sector is found. Then read backwards one sector at a time from the trailing edge of the block until a bad sector is found.
Then mark the bad sectors found (if any) as bad-sector, and mark the rest of the block as non-scraped without trying to read it. If any edge is already adjacent to a bad sector, it is considered as already trimmed and is not trimmed again.
3. Scraping phase
Scrape together the data not recovered by the copying or trimming phases. Scraping is done in one pass. Each non-scraped block is read forwards, one sector at a time. Any bad sectors found are marked as bad-sector.
4. Retrying phase (disabled by default)
Try to read again the bad sectors until the specified number of retry passes is reached. The direction is reversed after each pass. Every bad sector is tried only once in each pass.
ddrescue can’t know if a bad sector is unrecoverable or if it will be eventually read after some retries.
Next (2d 20h 26m and still rescuing) Lets read the source code!
How many of you are using some tools, and often find themselves saying:
How is this program working? How is it written? Can i write things like that? Is it magic? Hocus Pocus?Well, Only one day to find out! Download the source code, and start reading! There is not better way to learn how to write amazing tools using good filetree, how to write Documentation for a project, How to write Makefiles, How to separate Header files and actual code, how to … —===Enter your words of chaoice here===—, Other than reading a source code of some well establishes tools out there!
You can find the source code Here
And the project home HereI know! i Know! i have ddresuce-1.16 in the screenshot, and i have just pointed you to a website that has 1.25 and 1.26! and you are going to ask WHY? The answer is simple enough! I already had ddrescue-1.16 in a flash drive but since i decided to make this Post… here you go!
Tip #1, after you have read your first source code, Go back and download an earlier version, just to see what they have changes, and try to things WHY! or come back later, download the new version and see what they have changed! This is a very very good lesson as well!
Before i close the post for the day (i’m sure that i’ll open it tomorrow as well) Here is an updated screenshot of ddrescue terminal!
–==Now at Pass 2==–
I’m off to read some source code! See you … soon!
[1 Million Years Later]….
The disk have been almost recovered.
There was 99.76% recovery
0.22% left for scraping
0.01% bad sector
BUT!! In scraping phase (and the last Retrying phase) there was an ERROR:
Unaligned read error. Is sector size correct?
Therefore the was no point in having ddrescue brutalize the disk.
What is the next step?
First try to recover the disk so it can be mounted :
e2fsck -c /dev/sdg1And then just reboot to windows. You will see that the disk is mounted and your content is there!
THIS IS NOT THE END
There is one more step!!! Since the recovery is not 100% there is a need to do a chkdsk Check Disk from windows.
Step1: Open a terminal or Power shell with admin privileges
Step 2: Run cgkdsk <Disk letter> /f /R /X /B
Parameters
Parameter Description
<volume> Specifies the drive letter (followed by a colon), mount point, or volume name.
[ [<path>]<filename> Use with file allocation table (FAT) and FAT32 only. Specifies the location and name of a file or set of files that you want chkdsk to check for fragmentation. You can use the ? and * wildcard characters to specify multiple files.
/f Fixes errors on the disk. The disk must be locked. If chkdsk cannot lock the drive, a message appears that asks you if you want to check the drive the next time you restart the computer.
/v Displays the name of each file in every directory as the disk is checked.
/r Locates bad sectors and recovers readable information. The disk must be locked. /r includes the functionality of /f, with the additional analysis of physical disk errors.
/x Forces the volume to dismount first, if necessary. All open handles to the drive are invalidated. /x also includes the functionality of /f.
/i Use with NTFS only. Performs a less vigorous check of index entries, which reduces the amount of time required to run chkdsk.
/c Use with NTFS only. Does not check cycles within the folder structure, which reduces the amount of time required to run chkdsk.
/l[:<size>] Use with NTFS only. Changes the log file size to the size you type. If you omit the size parameter, /l displays the current size.
/b Use with NTFS only. Clears the list of bad clusters on the volume and rescans all allocated and free clusters for errors. /b includes the functionality of /r. Use this parameter after imaging a volume to a new hard disk drive.
/scan Use with NTFS only. Runs an online scan on the volume.
/forceofflinefix Use with NTFS only (must be used with /scan). Bypass all online repair; all defects found are queued for offline repair (for example, chkdsk /spotfix).
/perf Use with NTFS only (must be used with /scan). Uses more system resources to complete a scan as fast as possible. This may have a negative performance impact on other tasks running on the system.
/spotfix Use with NTFS only. Runs spot fixing on the volume.
/sdcleanup Use with NTFS only. Garbage collect unneeded security descriptor data (implies /f).
/offlinescanandfix Runs an offline scan and fix on the volume.
/freeorphanedchains Use with FAT/FAT32/exFAT only. Frees any orphaned cluster chains instead of recovering their contents.
/markclean Use with FAT/FAT32/exFAT only. Marks the volume clean if no corruption was detected, even if /f was not specified.
/? Displays help at the command prompt./f to fix errors
/R to locate bad sector and recover information
/X to force volume dismount
/B to Clears the list of bad clusters on the volume and rescans all allocated and free clusters for errors.
ARM your self with patience since this is going to take a little bit long
and then enjoy your new disk!!!!
Personally after this step I will copy everything to a new disk, and re-initialize both of my disk used for recovery.
Have fun and enjoy!
Found this helpful?