A common and befuddling problem with computers is the sudden and seemingly inexplicable disappearance of an external hard drive that has been functioning properly. It can be a breathtaking experience to suddenly be told that your data, often irreplaceable pictures and documents, might be gone forever.

As with many similar situations in life the appropriate response is “Don’t panic”. When approached sensibly and carefully, the situation can be resolved and the data saved more times than not. Here we will look at the causes, fixes and preventative measures for instances involving external drives being used with Windows XP and Windows Vista/7.

Error Messages

Windows XP	Windows 7
DO NOT FORMAT THE DRIVE

Properties of the drive show that both used and free spaces are 0 byte in size for the raw drive
Windows XP	Windows 7

What does RAW partition mean

Why does it happen and how to avoid it

How do you fix it

Do you want to recover anything extra important first?

Repair partition table and boot sector

In order to repair the most common problems that cause an external disk to suddenly become RAW we will use a PartedMagic Linux boot CD and one of the tools included on it called TestDisk. Both of these items are free and I urge you to donate to them if you are helped by their use and have the ability to make a donation.

This method allows us to work on the drive without attempting to mount it and gives us more direct access to it than trying to fix it in Windows.

First make a bootable CD or USB stick of PartedMagic. Instructions can be found here.

Boot the computer that has the bad drive connected to it with the PartedMagic CD/USB that you made. A menu will pop up, please select
1. Default Settings (Runs from RAM)

When it finishes booting you will notice that it looks kind of like a Windows desktop. The button in the lower left works like the START button in Windows XP and allows you to select and run various programs or shutdown the computer. To make things easy, I’ll just refer to this as the START button.

Click on the Terminal button down near the START button (It has a picture of a monitor on it) and type
testdisk
and press enter.

TestDisk is a console application so you have to use your keyboard to interact with it instead of your mouse.

Choose No Log and hit enter

Make sure that your external disk is highlighted
Choose Proceed and hit enter

Select Intel and hit enter (there is a slight chance that the partition is EFI GPT if the drive is 2TB or greater in size and at the bottom of this screen it says that Hint: EFI GPT was detected. If that is the case, select EFI GPT instead and try that.)

Select Analyse and hit enter

The partition data looks incorrect (an explanation of why is beyond the scope of this article)
Select Quick Search and hit enter

Say ‘Y‘ if it asks if the disk was made in Vista/Win7 (even if it was made in XP say ‘yes’)

Now the Quick Search will run

When the Quick Search completes, we see one partition (if you only see the option to CONTINUE at the bottom of the screen press ENTER one time to continue to the screen that you need to be at).

Lets look at the data on that partition press
P
and you should see a list of files/folders in the partition.

Hmmm… This looks like a bunch of diagnostic tools but not our missing data. We’ll need to look further. Press
Q
to go back a screen and then press
enter
To get to this screen, select DEEPER SEARCH and press enter.

Naturally, the Deeper Search takes longer than the Quick Search

When the Deeper Search completes we now see two partitions. The one we saw after the quick search and another one.

Select the new partition and press
P
to see the files/folders and now we see the data we want to make visible again.

Press
Q
to go back a screen and then press enter to get to this screen.
Select WRITE and press enter in order to write our new partition table to the drive.
DO NOT WRITE A NEW PARTITION TABLE IF YOU DID NOT SEE YOUR FILES/FOLDERS That means the proper partition table was not found and you will have to try a different technique to recover your data.

Press Y to confirm that you want to write the new partition table.

As I mentioned at the start of this repair, this disk also has corruption in the boot sector of the partition that we are trying to make visible again. When this is the case TestDisk will stop at this point and ask you to fix it before it finishes writing the new partition table. If the backup boot sector is good (as in this case) simply choose BACKUP BS and press enter. In the event that the backup boot sector is also bad then choose REBUILD BS.

Press Y to confirm the boot sector repair

Then choose QUIT and press enter

Regardless of whether a boot sector repair is required or not, you will end up at this screen where all you can do is press enter. Then hit Q until you are out of TestDisk.

Now remove the PartedMagic CD and shutdown the computer START>SHUTDOWN and then try booting it into Windows to see if the drive is now visible.

Here are the results for the drive used in this example…

Backup important files and run chkdsk

Now you should make backups of all important files on the drive in case it goes bad again and then run chkdsk to fix any other filesystem errors that may still exist.
After backing up important files, right-click the disk in Windows Explorer and choose PROPERTIES. Go to the TOOLS tab and click CHECK NOW. Check both options and click START.

Hopefully, this solves your problem. Please feel free to contact me through the contact form or comments if you have any questions or suggestions for this post.

• Hard disk absent
• Hard disk failed
• Hard disk missing
• Hard disk not found
• Hard disk error

• No Bootable Device Available
• No Bootable Devices Found
• Boot Device Not Found
• Error Insert proper boot device
• Reboot and select proper boot device error

Troubleshooting

• Invalid partition table.
• Error loading operating system.
• Missing operating system.
• No Bootable Devices Found
• Reboot and select proper boot device error

Common Boot Errors Bootloader to Logon

• Black Screen Hang (before splash screen shows)
• Disk Read Error occurred

• NTLDR missing
• NTLDR compressed

• Invalid Boot.ini
• Boot Device is Inaccesible
• Hal.dll missing or corrupt

• NTDETECT V1.0 Checking Hardware
• NTDetect Failed
• System Hive is Corrupt

• Windows could not start because of a computer disk hardware configuration problem.
• Could not read from the selected boot disk. Check boot path and disk hardware. (After verifying boot.ini is correct)
• Windows NT could not start because the following file is missing or corrupt: Winnt_root\System32\Ntoskrnl.exe
• Winnt_root\System32\Hal.dll missing or corrupt
• Black Screen Hang (after splash screen appears)

Solution Details

FULL DEMO Here

UPDATE 5-2-2013 Complete source code available for download here

DEMO of the first tab with source code Here

Poker hand analysis is an important tool for both the professional/learning player and for an online casino’s loss prevention staff.

The professional and learning players can use hand analysis to examine their own play and the pattern of play that their opponents may use.

The online casino’s loss prevention staff can use hand analysis to spot anomolies that may be indicative of collusion, cheating or other ‘advantage play’ situations.

The Problem

Both players and casino gaming survellance experts need to see the data representing the hand and the way it was played out in a concise, easy to follow format. A video takes far more time to watch than neccessary and anyone who has experience working with various ‘hand history’ files and formats knows that they are cumbersome and not consistent. The expert or educator wants to learn about or teach the hand without dealing with constant inconsistencies or searching through uneccessary data points. Time truly is money for this situation.

The Solution

Several years ago, poker computing legend, Mike Caro (http://en.wikipedia.org/wiki/Mike_caro) came up with a format for representing poker hands in a clear and concise way. He called it the MCU (Mike Caro University) Poker Chart which he published in his book “Caro’s Book of Poker Tells: The Psychology and Body Language of Poker” ISBN 1580420826. Here is a sample MCU Poker Chart generated from the program that I wrote which we will be discussing in this series of blog posts.

I’ve had the great pleasure of corresponding with Mr. Caro while working on this project and he is a very nice person in addition to being the somewhat reclusive genius he is portrayed as. I must disclose that I am a bit starstruck by Mr. Caro. He wrote the ORAC poker AI as well as Poker Probe back when I was just starting to learn to program computers.

I wanted to build a solution that can create MCU Poker Charts from varied datasets. Whether it be text file ‘hand histories’ direct database access or even live hands as they progress. The hand analyst need only feed it the data and then instantly start their work analyzing the play of the hand.

The solution that I settled on was a client side javascript application that runs on computers, laptops, tablets, phones, etc. using an HTML5 canvas that can be easily converted to an image file for saving or printing.

The first demo can be found here (go ahead and play with it) and uses a form for input. However, it would be trivial to replace the form with a database connection or the ability to load a hand history file into it. Here is an example output from a database where a specific hand is requested and the MCU Poker Chart is automatically generated and output.

First Version

For the initial version of the program we need to select a game that has a low complexity but is among the most popular games played. Once this works it is easy to expand it to cover other more complex games.
For this reason we chose
Limit Hold ‘em : No ante : 3 to 10 players

Using 3 to 10 players avoids the ‘forced bet’ difference of heads up (2 player) and using a limited bet version avoids the ‘side pot’ ‘allin’ betting of ‘no limit’. It is also a very popular game. ‘Forced betting’ simply consists of a small blind and big blind which moves with the button (dealer position). Betting amounts and order of betting are strictly governed by specific rules that are easy to simulate in our model.

Mr. Caro obviously put a lot of thought into what data should be included in the chart. I have discussed this with many people and a few more data elements have been brought up which are not in the chart. Here are some along with my reasons for not adding them in this version. Please provide any feedback or comments that you may have about this or other data that you feel is missing.

• Natural acumen (things that I feel an analyst sees automatically when they view the data in the chart and therefore would be redundant to include.)
  • Pot odds
    Players using pot odds can easily discern them from the data given. Pot odds become more of a ‘feeling’ for players that use them and are often only exactly calculated when they are in the gray area where the player has to decide “Do I gamble here?”
  • Strength of hand
    Like pot odds, a person who plays a lot starts to ‘feel’ the strength of their hand as cards are revealed. The chart displays betting patterns as cards are revealed so all of the information is there.
  • Tight / Loose play
    This is more of a trend that is determined over a series of hands by the same players. It is an important tool for determining the playing pattern of a potential opponent or for analyzing your own play. It is also an excellent indicator of collusion over a series of games. In the future this will probably be added to a version that analyzes multiple hands at one time and produces a trends graph.
• Algorithmic monitoring
  I’ve also been asked about data that I feel should be algorithmically detected by online casinos and is not really necessary for players to see. There is however, one of these data elements that is very important in analyzing live ‘in person’ play but not online play (except behind the scenes by algorithms). That is the timing of play.
  In a ‘real life’ game at a table there is a lot of knowledge to be gained by seeing a player agonize over a raise for longer than normal or quickly call. Online, this kind of timing is often useless because their phone could be ringing or their cat jumped on the keyboard. You don’t know.
  Algorithmic monitoring of this by the online casino can be used in conjunction with mouse and key-press tracking to detect bots. At which point further examination or a Turing test of some sort becomes appropriate. There is more to it than that but I don’t think it is wise or pertinent to this discussion to go into more detail.
  Since this application is being targeted to online play, timing is not included in this version. The “Add Note” feature on the form version can easily be used to add a footnote that describes an unusual timing.

So after boiling away all that appears to be extraneous data, we are left with Mr. Caro’s original chart. I think this speaks to how well thought out it is.

Again, I am alone in this project and would welcome any suggestions or arguments about the points I’ve made or missed completely.

Frameworks used.

I’d like to keep this free in the spirit that Mr. Caro wished it to be when he created it so all frameworks used are open source. Any of my custom code will also be open source.
This is important for the ability of the code to be reviewed. Especially if it is ever needed as evidence of a crime or fraud or to prevent its misuse for such.

I’ve opted to keep as much of this as possible on the client side using Javascript so that the creation of the chart occurs on the user’s device and not on a server that they have no control over.

Cross-browser Compatibility (works in all common browsers on all devices)
JQuery http://jquery.com/
JQuery-UI (for the form version) http://jqueryui.com/
Google Chrome Frame add-on https://developers.google.com/chrome/chrome-frame/ (for compatability with older Internet Explorer browsers)

Source Code of the first tab

The two main features of the MCU Poker Chart Generator are

1. It runs on all devices and operating systems.
2. It works off of all the different data sources out there for hand histories with minimal customization.

These goals are met by using a client-side Javascript solution.

Simply put, the application.

Gets the data.
Draws the canvas.
Draws the data in the proper positions on the canvas.

The demo uses a fake form (DIV with input elements) for data input and this is accessed by the application using the DOM. It could just as easily be accessing a JSON data set or XML or data straight from any database even straight from a text hand history file.

Let’s take a look at the code that grabs the data from the first tab of the form and uses it to start placing data on the canvas.

Go here to see single tab demo and source code

DEMO of the first tab with source code Here

UPDATE 5-2-2013 Complete source code available for download here

FULL DEMO Here

Continuing with this method for the other tabs creates an application that can take data from nearly any source and display an MCU Poker Chart for the hand on any device.

It is easy to modify for any poker game. If you’d like more information on creating something like this for your website or business, contact me.

In the course of repairs that my clients ask me to perform, occasionally I am confronted with a CD or DVD that they wish to get the data from but are having difficulty.

There are many reasons that this can happen such as scratches, dirt, ink seeping through from the label or even cracks.
IF THE CD/DVD IS CRACKED, BE PREPARED FOR THE POSSIBILITY THAT IT MAY SHATTER IN THE DRIVE. SINCE TIME MAY BE LIMITED, YOU PROBABLY WANT TO MOVE STRAIGHT TO ATTEMPTING TO IMAGE THE DISC AND HANDLE IT AS LITTLE AS POSSIBLE.
I have lost two optical drives to exploding discs. Both times the discs were badly cracked and attempts were made to limit the speed and maintain the best possible balance but they still exploded. Fortunately a large amount of data (all of it in one case) was recovered prior to the disc self destructing.

Normally though, I encounter discs that are scratched and scuffed to the point that they are no longer readable. Often you can view the disc and the files on it but any attempts to open or read the files gives an error.

Let’s make a mess.

First, we’ll make a CD with some public domain documents, video and audio files.

Then we’ll damage it by slicing the surface with a knife held at an angle to simulate really bad scratches.

Clean it

Let me to repeat the main rule of cleaning.
DO NOT MAKE THE DAMAGE WORSE
You are removing foreign material, not altering the physical state of the disk.
You are not trying to buff out scratches.

Try reading the disc again
MAKE SURE THE DISC IS COMPLETELY DRY
Sometimes cleaning is enough to give you an opportunity to copy all of the important files from the disc.
If you have access to multiple optical drives try them all. They can differ in quality and I have found some to be excellent at reading damaged discs that others simply won’t read.

Make a rescue image of the disc

Tools

If a free tool helps you, please donate to the people who created it.

PartedMagic by Patrick J. Verner (Free)

partedmagic.com

GNU ddrescue (Free, included in PartedMagic)

To get started, download and burn a PartedMagic BootCD/DVD and boot the computer with it.

In this example, we’ll make the image on a USB flash drive plugged into the computer so we will need to mount the partition on that which is, in our case, /media/sdd1. We’ll use the MOUNT DEVICES button in the lower left hand corner of the desktop to open Mount-gtk and then press the MOUNT button next to /media/sdd1.

Make an image with GNU ddrescue
Now we can open a Terminal window by clicking the monitor icon next to the MOUNT DEVICES icon in the lower left corner of the desktop.

We will do this in at least two passes. The first one will simply get all of the data that is easy to read and in good shape. For this pass we type in the command…

ddrescue -n -b 2048 /dev/sr0 /media/sdd1/mystuff.iso /media/sdd1/mystuff.log

ddrescue : The ddrescue command 
-n       : Tells it not to split failed blocks which would dig 
deeper into the problem areas but it does still attempt to trim 
failed blocks.

-b 2048  : Sets the block size to 2048 which is the standard 
block size for CD/DVD

/dev/sr0 : This is the optical drive with the bad CD in it 
(other versions of Linux may call this something else such as 
/dev/cdrom)

/media/sdd1/mystuff.iso : The recovered ISO file which will 
be saved on our flash disk.

/media/sdd1/mystuff.log : The log file which will be saved on 
our flash disk and will allow us to stop and resume the recovery 
whenever we wish without having to start from the beginning again.

It finishes in about 20 minutes but it does not look good. While it recovered about 310MB of data, it was not able to recover 155MB so we are still missing a third of our data.

At this point, we can try mounting that ISO and see what is on it. In this case, I found many files were intact but some of the audio books had large gaps in them or would not play at all even though they appeared to be the right size (since ddrescue fills the errors with empty spaces).

You can go through the files and if the important files that you want are intact and not damaged then you can recover them and you are done. However, if like in my example, the files you want to recover are damaged then you need to continue on to the second pass. It is often much more time consuming than the first pass was.

Second Pass

This time in the terminal we type

ddrescue -d -b 2048 /dev/sr0 /media/sdd1/mystuff.iso /media/sdd1/mystuff.log

After 24 hours (almost to the minute) it finished but was still unable to get 21MB of the data. That is a lot better than the 155MB that was missing after the first pass. When we mount the image, everything is fine except the cartoon. It is still messed up and corrupted in some parts. It is 86MB so 21MB is about a quarter of it. Playing it works but there are sections where the image or audio is corrupted and scrambled.

Try another CD/DVD player to get even more

Third Pass

I have a very good external optical drive that I can use to try to get some more of that data. Here we hook it up and attempt a third pass using the same command as we did for the second pass (we change the source path to the external optical drive) and we add -r 3 to have it retry all the data that it could not get before three more times.

ddrescue -d -r 3 -b 2048 /dev/sr1 /media/sdd1/mystuff.iso /media/sdd1/mystuff.log

After 24 hours that only got us another 2.2MB so there is still about 18MB missing. It never even finished completing the first retry in that time.

Fourth Pass

Next we try a standard internal drive from an old broken computer that we hook up externally with a PATA/USB bridge. I don’t expect it to do much but let’s try it.

ddrescue -d -r 3 -b 2048 /dev/sr1 /media/sdd1/mystuff.iso /media/sdd1/mystuff.log

Wow! After 24 hours an additional 5MB of the worst damage is recovered so there is only 13MB left that is a recovery of 97.2% of the data. We’ll let this one run for a while longer. After another 12 hours we get another 2MB and that seems to be the extent of our recovery efforts with these tools and drives.

First pass 20 minutes 155MB missing out of 470MB 33% data lost
Second pass 24 hours 21MB missing out of 470MB 4.5% data lost
Third pass 24 hours 18MB missing out of 470MB 3.8% data lost
Fourth pass 36 hours 11MB missing out of 470MB 2.3% data lost

97.7% recovered after three and a half days. I tried several more optical drives and also retried drives that I had already used but could only get less than 100Kb more so this appears to be the best we can do with this technique. All of the data is playable, viewable and readable except for some periods of audio and video corruption in the cartoon.

1.0 Introduction

Being able to effectively analyze the S.M.A.R.T. data on a hard disk drive (HDD) enables you to quickly identify problems that can aid you in recovering all of the data from it before it becomes irretrievable or requires significant expense to retrieve.

The vast majority of hardware repairs that I do for clients involve problems with hard disk drives. If the computer’s BIOS sees the drive okay and it is not making any unusual sounds, the first thing I do is examine the S.M.A.R.T. data on the drive.

S.M.A.R.T. stands for Self-Monitoring, Analysis and Reporting Technology and is the hard drive’s record of its internal diagnostic monitoring and usage statistics packaged for being accessed externally. The primary purpose of S.M.A.R.T. is to alert us to an impending failure of the drive while there is still time to save the data. When a hard drive reports that the S.M.A.R.T. health is FAILED you must get the important data off of it immediately and not use the drive anymore.

IT IS IMPORTANT TO UNDERSTAND THAT DIFFERENT MANUFACTURERS AND DIFFERENT DRIVE MODELS CAN STORE AND DISPLAY DIFFERENT SUBSETS OF S.M.A.R.T. DATA AND CALCULATE VALUES AND TOLERANCES DIFFERENTLY.

S.M.A.R.T. data is best used as a general guide. For specifics, the hard drive’s technical documentation must be consulted. Sometimes that information is difficult to come by and can be proprietary. For example, the formulas for how some manufacturers calculate normalized values for attributes can be very difficult to find.

Wikipedia entry for S.M.A.R.T.

After making sure that the connection between the drive and the computer is correct and its quality is the best it can be, you need to check the health of the drive itself. Then you can rule out physical damage that requires repair with specialized tools in a controlled environment.

It is best to do this in Linux because Windows will try to mount the drive’s file systems automatically and that may add layers of abstraction that make it difficult to access the health statistics on the drive. A Linux LiveCD or boot CD is fine.

1.1 Drive Health Check Summary Using S.M.A.R.T. Data

·        Does the drive make noises that it did not make before?

o   Loud clicking or a periodic beeping sound may indicate mechanical damage. Physical damage may require the drive to be sent to a facility with the proper tools and equipment to recover the data.

·        Does S.M.A.R.T. show any errors or bad attribute values?

o   See section 1.2 below for details on checking S.M.A.R.T. attributes with ID#s 1, 5 and 197.

·        Does it pass the S.M.A.R.T. Short Self-Test?

o   See section 1.3 below for details on how to run the S.M.A.R.T. Short Self-Test.

If it does not pass any of these tests, you need to either get important files off of the drive immediately (if you can access them) or make an image of the drive (see “Making an image with ddrescue” for how to make an image of a failing drive) and attempt recovery of data from the image.

Tools

If a free tool helps you, please donate to the people who created it.

GSmartControl by Alexander Shaduri (using smartctl by Bruce Allen) (Free)

gsmartcontrol.berlios.de

PartedMagic by Patrick J. Verner (Free)

partedmagic.com

1.2 Check the S.M.A.R.T. Data on the Drive

To get started, download and burn a PartedMagic BootCD/DVD and boot the computer with it.

partedmagic.com

The GUI version of GSmartControl is preinstalled in PartedMagic and appears on the desktop as an icon that says DISK HEALTH beneath it. (GSmartControl can also be run from Windows but it may be more difficult to access a failing drive due to automated attempts to mount file systems on the drive)

In this example, we’ll look at a Western Digital Scorpio Blue hard drive that is known to have errors. We select our drive and the results of the ATA interface command IDENTIFY DEVICE are displayed on the first tab. Notice that at the bottom it says that it has passed the Self-Assessment test. This drive has errors but not bad enough ones for it to fail S.M.A.R.T.

The Attributes tab gives a quick grid view of the S.M.A.R.T. attributes that are tracked and reported by this drive and their data. We’ll discuss these later when we look at the report we create for this drive. As you can see, some of the lines are highlighted pink in order to bring attention to potential error data.

The Capabilities tab tells us what S.M.A.R.T. features this hard drive has available, including self-tests that it can do and how it periodically saves the S.M.A.R.T. data.

The Error Log tab shows the error log data that is being stored in the hard drive’s S.M.A.R.T. and usually only contains the last five errors.

The Self-Tests Log tab shows the results of any Self-Tests that have been run.

The Perform Tests tab has the options for you to have it perform any of the S.M.A.R.T. self-tests available for that hard drive.

Click the SAVE AS button at the bottom of the screen to save the S.M.A.R.T. report for the drive. Make sure that you have mounted a drive to save it to first otherwise it will save it to the RAM disk which is in memory and will disappear as soon as you shut down the computer. The easiest way to mount a drive in PartedMagic is to click the MOUNT DEVICES button in the lower left corner of the desktop.

Let’s look at our report.

First we see some of the information that the hard drive reports when it is sent the ATA command IDENTIFY DEVICE. We also see if the drive is in smartctl’s database of drives so it knows if there are any special considerations for interpreting the S.M.A.R.T. data for that drive.

smartctl 5.42 2011-10-20 r3458 [i686-linux-3.4.4-pmagic] (local build)
Copyright (C) 2002-11 by Bruce Allen, 

http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Scorpio Blue Serial ATA
Device Model:     WDC WD1200BEVS-75RST0
Serial Number:    WD-WXE607356256
LU WWN Device Id: 5 0014ee 2005c2a3e
Firmware Version: 04.01G04
User Capacity:    120,034,123,776 bytes [120 GB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   7
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Fri Nov 16 09:31:04 2012 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Next is the drive’s health assessment. This is followed by details of the S.M.A.R.T. capabilities that this drive has.

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Disabled.
Self-test execution status:      ( 112)	The previous self-test completed having
					the read element of the test failed.
Total time to complete Offline 
data collection: 		( 5400) seconds.
Offline data collection
capabilities: 			 (0x7b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine 
recommended polling time: 	 (   2) minutes.
Extended self-test routine
recommended polling time: 	 (  70) minutes.
Conveyance self-test routine
recommended polling time: 	 (   6) minutes.
SCT capabilities: 	       (0x103f)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

Next the attributes that are monitored by this drive are listed. This is a pretty common subset of attributes listed in our report here but you will come across others. For example, Hitachi drives often have a “G-Sense Error Rate” attribute that uses accelerometers inside the drive to detect and record any strong shocks if the drive is dropped or bumped very hard.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   192   190   051    Pre-fail  Always       -       54689
  3 Spin_Up_Time            0x0003   187   185   021    Pre-fail  Always       -       1633
  4 Start_Stop_Count        0x0032   096   096   000    Old_age   Always       -       4058
  5 Reallocated_Sector_Ct   0x0033   193   193   140    Pre-fail  Always       -       49
  7 Seek_Error_Rate         0x000e   143   143   051    Old_age   Always       -       3023
  9 Power_On_Hours          0x0032   086   086   000    Old_age   Always       -       10887
 10 Spin_Retry_Count        0x0012   100   100   051    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0012   100   100   051    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   097   097   000    Old_age   Always       -       3653
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       88
193 Load_Cycle_Count        0x0032   162   162   000    Old_age   Always       -       115356
194 Temperature_Celsius     0x0022   120   078   000    Old_age   Always       -       27
196 Reallocated_Event_Count 0x0032   174   174   000    Old_age   Always       -       26
197 Current_Pending_Sector  0x0012   175   167   000    Old_age   Always       -       506
198 Offline_Uncorrectable   0x0010   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   051    Old_age   Offline      -       0

Interpreting attributes

Attributes can be calculated differently by different manufacturers so here are some general rules to guide you. Let’s look at our drive’s values for attribute with ID# 1.

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   192   190   051    Pre-fail  Always       -       54689

ID#

This ID for the attribute is displayed as a decimal number in this report but other reports may display it in its native hexadecimal format.

Attribute Name

The human readable description of the ID#

Flag

This is a hexadecimal description of the purposes of this attribute combined so they can be represented by a number. For example is it for performance, error rates or pre-fail or some combination of these and others.

Value

This is a normalized version of the data that starts at its best possible value (like 200 for this attribute) and then decreases as it gets worse (now 192 for this attribute).

Worst

The worst that the Value has ever been.

Threshold (THRESH)

The threshold for the Value. Once the Value goes below this, the drive will consider it to be FAILED for this attribute and depending on the attribute (like this one) the whole drive will be considered to be failing and warnings will be sent to backup and replace it.

Type

A human readable version of the Flag value but not as detailed.

Updated

How often the attribute’s value is updated.

When Failed

If there was a failure where the value reached or went below the threshold value, the power-on lifetime is displayed here (usually in hours )

Raw Value

The raw non-normalized value of the attribute.

The attributes that you should pay careful attention to are ID#s 1, 5 and 197 (01, 05, C5 in hexadecimal). These are the most common attributes with problems that cause a recoverable disappearance of drives.

ID#1 or 01 ‘Raw Read Error Rate’ is related to the frequency of errors appearing while reading RAW data from a disk. Ignore the Raw Value and go by the normalized Value. If it is decreasing and getting close to the threshold value then the drive can probably be recovered by making an image of it (see “Making an image with ddrescue” for how to make an image of a failing drive) but it should then be replaced after the data is retrieved from it.

ID#5 or 05 ‘Reallocated Sector Count’ is the quantity of remapped sectors. This is the number of ‘hard’ bad sectors that have been reallocated by the internal controls of the hard drive. The hard drive has a number of sectors in reserve to replace these bad ones. These are inaccessible sectors that the drive expects to be bad forever which means they are most likely physically damaged. Data that was in those sectors was probably lost for good. Modern hard drives should not have any. My personal ‘rule of thumb’ is that if there are less than 10 of these (Raw Value is less than 10) and after checking it periodically during use I find that the number is not increasing, I then believe the drive to be trustworthy for future use but continued monitoring of the value is recommended. That’s my personal rule but I would not fault anyone who replaces a drive because there is just one of these.

ID#197 or C5 ‘Current Pending Sector’ is the quantity of sectors that are not able to be read but have not been reallocated yet. These are ‘soft’ bad sectors because there is still a chance that they will be successfully read. The way they typically become ‘hard’ bad sectors is when an attempt is made to write to them and is unsuccessful. When this happens, they are removed from this attribute and added to ID#5. When this happens a new sector is pulled from the reserve pool of sectors on the hard drive and used to replace this one. Many programs proclaim that they can repair bad sectors. What they often do is not really repair them. Instead they force the drive to attempt to write to them and if they are unable to it forces them to be reallocated with sectors from the reserve pool and to the computer user, it appears as if the ‘soft’ bad sector has been repaired. In fact, the data in the sector has been lost and it has been replaced with a new and healthy sector. I typically treat this number the same as ID#05. My personal ‘rule of thumb’ is that if there are less than 10 of these (Raw Value is less than 10) and after checking it periodically during use I find that the number is not increasing, I then believe the drive to be trustworthy for future use but continued monitoring of the value is recommended. That’s my personal rule but I would not fault anyone who replaces a drive because there is just one of these.

Next in our report is the data from the S.M.A.R.T. data’s error log and the results of any self-tests that have been performed. You can identify if the errors have occurred recently by looking at the timestamp on the error.

Error 58074 occurred at disk power-on lifetime:10886 hours (453 days + 14 hours)

In our case the most recent error occurred at 10886 power-on lifetime hours and the report we ran was at 10887 power-on lifetime hours according to the value of attribute 09.

9 Power_On_Hours  0x0032 086   086   000  Old_age   Always  -  10887

So the error occurred within the last hour of it being powered on and is recent.

SMART Error Log Version: 1
ATA Error Count: 58074 (device log contains only the most recent five errors)
	CR = Command Register [HEX]
	FR = Features Register [HEX]
	SC = Sector Count Register [HEX]
	SN = Sector Number Register [HEX]
	CL = Cylinder Low Register [HEX]
	CH = Cylinder High Register [HEX]
	DH = Device/Head Register [HEX]
	DC = Device Command Register [HEX]
	ER = Error register [HEX]
	ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 58074 occurred at disk power-on lifetime: 10886 hours (453 days + 14 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 da da d7 ed  Error: UNC at LBA = 0x0dd7dada = 232250074

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  20 00 08 da da d7 0d 00      13:53:18.903  READ SECTOR(S)
  20 00 01 00 00 00 00 00      13:53:18.902  READ SECTOR(S)
  c6 00 00 da da d7 00 00      13:53:18.902  SET MULTIPLE MODE
  20 00 80 da da d7 0d 00      13:53:15.871  READ SECTOR(S)
  20 00 08 ba 78 da 0d 00      13:53:15.868  READ SECTOR(S)

Error 58073 occurred at disk power-on lifetime: 10886 hours (453 days + 14 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 80 da da d7 ed  Error: UNC at LBA = 0x0dd7dada = 232250074

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  20 00 80 da da d7 0d 00      13:53:15.871  READ SECTOR(S)
  20 00 08 ba 78 da 0d 00      13:53:15.868  READ SECTOR(S)
  30 00 08 ba 78 da 0d 00      13:53:15.864  WRITE SECTOR(S)
  40 00 08 ba 78 da 0d 00      13:53:15.864  READ VERIFY SECTOR(S)
  20 00 01 00 00 00 00 00      13:53:15.862  READ SECTOR(S)

Error 58072 occurred at disk power-on lifetime: 10886 hours (453 days + 14 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 05 d5 da d7 ed  Error: UNC at LBA = 0x0dd7dad5 = 232250069

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  20 00 08 d2 da d7 0d 00      13:53:12.988  READ SECTOR(S)
  20 00 01 00 00 00 00 00      13:53:12.988  READ SECTOR(S)
  c6 00 00 d5 da d7 00 00      13:53:12.988  SET MULTIPLE MODE
  20 00 80 d2 da d7 0d 00      13:53:09.796  READ SECTOR(S)
  20 00 08 b2 78 da 0d 00      13:53:09.793  READ SECTOR(S)

Error 58071 occurred at disk power-on lifetime: 10886 hours (453 days + 14 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 7d d5 da d7 ed  Error: UNC at LBA = 0x0dd7dad5 = 232250069

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  20 00 80 d2 da d7 0d 00      13:53:09.796  READ SECTOR(S)
  20 00 08 b2 78 da 0d 00      13:53:09.793  READ SECTOR(S)
  20 00 08 c2 da d7 0d 00      13:53:09.790  READ SECTOR(S)
  20 00 08 b2 78 da 0d 00      13:53:09.787  READ SECTOR(S)
  30 00 08 b2 78 da 0d 00      13:53:09.784  WRITE SECTOR(S)

Error 58070 occurred at disk power-on lifetime: 10886 hours (453 days + 14 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 07 cb da d7 ed  Error: UNC at LBA = 0x0dd7dacb = 232250059

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  20 00 08 ca da d7 0d 00      13:53:06.877  READ SECTOR(S)
  20 00 01 00 00 00 00 00      13:53:06.876  READ SECTOR(S)
  20 00 08 c2 da d7 0d 00      13:53:06.874  READ SECTOR(S)
  20 00 01 00 00 00 00 00      13:53:06.873  READ SECTOR(S)
  c6 00 00 cb da d7 00 00      13:53:06.873  SET MULTIPLE MODE

Next we see the log of the results from any self-tests that have been run

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed: read failure       50%     10887         232252410
# 2  Extended offline    Completed: read failure       90%     10886         186194165
# 3  Conveyance offline  Completed: read failure       90%     10886         186194165
# 4  Short offline       Completed: read failure       90%     10886         186194165
# 5  Short offline       Completed: read failure       90%     10872         186178051
# 6  Short offline       Completed without error       00%         1         -
# 7  Extended offline    Completed without error       00%         1         -
# 8  Short offline       Completed without error       00%         0         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

1.3 S.M.A.R.T. Self-Tests

Most S.M.A.R.T. capable drives have several self-tests that you can ask them to perform. For example, in GSmartControl you can find and execute the available tests on the PERFORM TESTS tab.

S.M.A.R.T. Self Tests (Not available on all drives)

·        Conveyance Test

o   Intended as a quick test to identify damage incurred during transporting of the device from the drive manufacturer to the computer manufacturer.

·        Short Test

o   Checks the electrical and mechanical performance as well as the read performance of the disk. This is the test that you want to run. If it fails, make an immediate image of the drive (see “Making an image with ddrescue” for how to make an image of a failing drive).

·        Long Test

o   A longer and more thorough version of the short self-test. It scans the entire disk surface, with no time limit. If the drive has a little damage, this test may make it worse. Do not run it until the important data on the drive has been recovered and backed up elsewhere.

·        Selective Test

o   Some drives allow selective self-tests of just a part of the surface.

The drive in this example had to have an image made of it (see “Making an image with ddrescue” for how to make an image of a failing drive) in order to recover the data from it. It was a successful recovery but the drive was no longer trustworthy for continued use.

Most of the time you can use standard registry editing tools on a Windows computer that cannot boot into Windows.

Using chntpw as a Windows registry editor