Recent

Author Topic: SD card low level read  (Read 5901 times)

dseligo

  • Hero Member
  • *****
  • Posts: 1196
SD card low level read
« on: August 25, 2021, 12:47:03 pm »
How could SD card be read if there isn't drive letter assigned to it and it doesn't show in Disk Management?

I can read cards that show in Disk Management with this:
Code: Pascal  [Select][+][-]
  1. hBlockDevice := CreateFile(PChar(sFileName), GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, 0, 0);
  2. // where sFileName is something like \\.\PHYSICALDRIVE1
  3.   if hBlockDevice <> INVALID_HANDLE_VALUE then
  4.   begin
  5.     SetFilePointer(hBlockDevice, BlockSize * dwSector, nil, FILE_BEGIN);
  6.     ReadFile(hBlockDevice, Buffer[0], BlockSize, dwRead, nil);
  7.     ...

But if card doesn't show in Disk Management I get INVALID_HANDLE_VALUE.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: SD card low level read
« Reply #1 on: August 25, 2021, 08:56:49 pm »
In the general case, forget it. BTDT, and the only way you can get low-level access to a possibly-damaged card is by using something like an Arduino.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

winni

  • Hero Member
  • *****
  • Posts: 3197
Re: SD card low level read
« Reply #2 on: August 25, 2021, 09:08:48 pm »
Hi!

There is a marvelous tool  simply called testdisk

https://www.cgsecurity.org/wiki/TestDisk_Download


Available for a lot of OS but I would advise Linux, because Windows gives up quick.

As long a Linux detects a device there is a good chance to recover your data.

Before you plug the card do a

ls /dev/sd*

Plug the card and do the same. If something changed then the new device is your card. Then there is  good chance to get the data back.

I recovered a lot of broken hard-disks with testdisk. For example:

*External 350 GB full of music but with a zerod  directory
*Laptop-Disk, which had a big scratch from a crash: 85% of the photos saved. The other 15% were on the cratch

Hope you are lucky!
 
Winni

dseligo

  • Hero Member
  • *****
  • Posts: 1196
Re: SD card low level read
« Reply #3 on: August 26, 2021, 12:15:38 am »
Cards are not damaged. They are used in microcontroller operated devices and probably has some custom memory organization.
I hoped I could read them from Windows when some of this devices stop working.

I will try with Linux and if that doesn't work I'll try it with Arduino.

dseligo

  • Hero Member
  • *****
  • Posts: 1196
Re: SD card low level read
« Reply #4 on: August 26, 2021, 12:22:01 am »
There is a marvelous tool  simply called testdisk

https://www.cgsecurity.org/wiki/TestDisk_Download

Maybe it won't help me now with this SD cards, but thank you for tip on this tool. I just hope I won't need it anytime soon. :)

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: SD card low level read
« Reply #5 on: August 26, 2021, 08:12:09 am »
Cards are not damaged. They are used in microcontroller operated devices and probably has some custom memory organization.
I hoped I could read them from Windows when some of this devices stop working.

I will try with Linux and if that doesn't work I'll try it with Arduino.

My experience is that when they go wrong, they go wrong badly. Getting the "physical damage" side out of the way first, the narrowest line across the card runs straight through the die so if they crack- and I've done this twice due to a badly-designed Raspberry Pi case- they're beyond even a dissolve-card-and-rebond solution.

Electrically, they're known to be very susceptible to poor voltage control and power interruption at critical times. IIRC, the low-level metadata blocks are R/O so won't be damaged by this sort of thing, but it's still very easy to get to a situation where the card doesn't respond normally and you have to go in using SPI: hence Arduino although in principle at least any SPI controller e.g. a Raspberry Pi would work.

Logically, since they don't have load balancing they're extremely susceptible to having their FAT-style directory screwed by repeated write operations. Now at least some cards are specially designed on the assumption that they'll be partitioned in a way that puts the FATs in a specially-designed resistant area, but there's always a risk that any derangement to this will get the card into a state that a standard reader will reject it: and there's neither BIOS support nor something that looks like a (S)ATA controller to get in at a lower level. And of course most readers are "smart" with either a USB interface or a controller chip with a special Windows driver... and I've had no success at all getting the latter to work on Linux.

Oh, and if in doubt always check with a different (newer) reader: there's an arms race as capacities are pushed up by consumer equipment with PCs rather struggling to keep up.

HTH, MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

geraldholdsworth

  • Full Member
  • ***
  • Posts: 195
Re: SD card low level read
« Reply #6 on: June 19, 2022, 11:39:47 am »
How could SD card be read if there isn't drive letter assigned to it and it doesn't show in Disk Management?

I've been told that an SD card appears as a file on Linux, macOS and Windows. But, I have no clue what this is or how to access it.

I have the same query as you, about accessing an SD card that the OS doesn't recognise the format but is not broken.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: SD card low level read
« Reply #7 on: June 19, 2022, 12:10:18 pm »
How could SD card be read if there isn't drive letter assigned to it and it doesn't show in Disk Management?

I've been told that an SD card appears as a file on Linux, macOS and Windows. But, I have no clue what this is or how to access it.

I can speak for Linux and the same probably applies to the Mac.

Whether an SD-Card appears as a named device, to which fdisk may optionally be applied, and with the result mountable as a filesystem hence able to support files, depends on what comes between the card and the host computer.

If you have a USB-connected reader, there will be a chip or a "black blob" inside it. In either case that will be a microcontroller which presents itself as a USB storage device at one side, and on the other side talks either SPI or the MMC protocol (see Wp page).

If the OS doesn't recognise the card as a storage device, then you need to break in at the engineering level using either SPI or MMC. A PC is not well-suited for that since you need something like a CH341A chip to generate SPI to a (dumb) card adapter, a Raspberry Pi is somewhat better since it supports SPI directly.

However there are /far/ more people doing this sort of thing using hardware comparable with an Arduino, I have in the past used Arduino code in an attempt to see if there was any life at all left in a card that was physically damaged.

I'd also note that over the last couple of weeks I've been using an SD-Card attached to an RP2040: once I sorted out the non-standard pinning used by the prototyping board I'd got talking to the card using SPI was no real problem.

Quote
I have the same query as you, about accessing an SD card that the OS doesn't recognise the format but is not broken.

This is the second or third time I've said this in this thread: look at it using something like an Arduino since you're likely to need to get at it at a lower level than off-the-shelf PC interfaces support. Or at the very least use an RPi rather than a PC, but at that point expect to have to port Arduino code onto it since that's where all the work is being done.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

geraldholdsworth

  • Full Member
  • ***
  • Posts: 195
Re: SD card low level read
« Reply #8 on: June 20, 2022, 11:18:12 am »
This is the second or third time I've said this in this thread: look at it using something like an Arduino since you're likely to need to get at it at a lower level than off-the-shelf PC interfaces support. Or at the very least use an RPi rather than a PC, but at that point expect to have to port Arduino code onto it since that's where all the work is being done.
Unfortunately, people have requested a feature on my application that allows them to access the files on this filing system (which isn't recognised by Windows or macOS, but apparently there is a driver built into Linux...interestingly). The partition they want to access is a RISC OS partition. Majority of the time, this SD card would have come from a RaspberryPi, but sometimes a RiscPC (or older).

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: SD card low level read
« Reply #9 on: June 20, 2022, 11:25:14 am »
This is the second or third time I've said this in this thread: look at it using something like an Arduino since you're likely to need to get at it at a lower level than off-the-shelf PC interfaces support. Or at the very least use an RPi rather than a PC, but at that point expect to have to port Arduino code onto it since that's where all the work is being done.
Unfortunately, people have requested a feature on my application that allows them to access the files on this filing system (which isn't recognised by Windows or macOS, but apparently there is a driver built into Linux...interestingly). The partition they want to access is a RISC OS partition. Majority of the time, this SD card would have come from a RaspberryPi, but sometimes a RiscPC (or older).

Does the /device/ appear as e.g. /dev/sdb when plugged in? Note that I'm not talking about the /filesystem/ here, or about individual /files/ on the filesystem.

If it does, then it's a standard Linux question of (a) whether any form of partitioning is involved (which might or might not be directly supported) and (b) whether the kernel supports RISCOS filesystems.

This is /very/ different from having a card which the OS doesn't recognise, Google for something like 'mount RISCOS on Linux'.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

PascalDragon

  • Hero Member
  • *****
  • Posts: 5448
  • Compiler Developer
Re: SD card low level read
« Reply #10 on: June 20, 2022, 01:28:41 pm »
This is the second or third time I've said this in this thread: look at it using something like an Arduino since you're likely to need to get at it at a lower level than off-the-shelf PC interfaces support. Or at the very least use an RPi rather than a PC, but at that point expect to have to port Arduino code onto it since that's where all the work is being done.
Unfortunately, people have requested a feature on my application that allows them to access the files on this filing system (which isn't recognised by Windows or macOS, but apparently there is a driver built into Linux...interestingly). The partition they want to access is a RISC OS partition. Majority of the time, this SD card would have come from a RaspberryPi, but sometimes a RiscPC (or older).
Even if the SD card has an unsupported file system or even partition table it will still display as a drive in Window's disk manager (cause you might want to reformat it after all). If it doesn't then something is problematic with either the SD card or the reader.

dseligo

  • Hero Member
  • *****
  • Posts: 1196
Re: SD card low level read
« Reply #11 on: June 20, 2022, 07:08:05 pm »
This is the second or third time I've said this in this thread: look at it using something like an Arduino since you're likely to need to get at it at a lower level than off-the-shelf PC interfaces support. Or at the very least use an RPi rather than a PC, but at that point expect to have to port Arduino code onto it since that's where all the work is being done.
Unfortunately, people have requested a feature on my application that allows them to access the files on this filing system (which isn't recognised by Windows or macOS, but apparently there is a driver built into Linux...interestingly). The partition they want to access is a RISC OS partition. Majority of the time, this SD card would have come from a RaspberryPi, but sometimes a RiscPC (or older).
Even if the SD card has an unsupported file system or even partition table it will still display as a drive in Window's disk manager (cause you might want to reformat it after all). If it doesn't then something is problematic with either the SD card or the reader.

No. I have these devices which initialize SD card and after that cards aren't recognized in Windows or Linux at all. Cards work just fine in the device (device writes data to it and it can read data from card).

mas steindorff

  • Hero Member
  • *****
  • Posts: 532
Re: SD card low level read
« Reply #12 on: June 20, 2022, 10:59:23 pm »
I believe dseligo has an SD card that lacks a standard FAT table.  it takes 3x the writes to the SD card to properly support FAT32/FAT16 so for both speed and power reasons a lot of firmware just uses something else.
I have seen utilities On both Windoes paid software (winhex?) and Unix free utility (??) that can access the card on a block level, bypassing the OS file manager and the need for a good FAT table on the card.
Can anyone provide better help with this info?
windows 10 &11, Ubuntu 21+ - fpc 3.0.4, IDE 2.0 general releases

PascalDragon

  • Hero Member
  • *****
  • Posts: 5448
  • Compiler Developer
Re: SD card low level read
« Reply #13 on: June 21, 2022, 09:29:43 am »
This is the second or third time I've said this in this thread: look at it using something like an Arduino since you're likely to need to get at it at a lower level than off-the-shelf PC interfaces support. Or at the very least use an RPi rather than a PC, but at that point expect to have to port Arduino code onto it since that's where all the work is being done.
Unfortunately, people have requested a feature on my application that allows them to access the files on this filing system (which isn't recognised by Windows or macOS, but apparently there is a driver built into Linux...interestingly). The partition they want to access is a RISC OS partition. Majority of the time, this SD card would have come from a RaspberryPi, but sometimes a RiscPC (or older).
Even if the SD card has an unsupported file system or even partition table it will still display as a drive in Window's disk manager (cause you might want to reformat it after all). If it doesn't then something is problematic with either the SD card or the reader.

No. I have these devices which initialize SD card and after that cards aren't recognized in Windows or Linux at all. Cards work just fine in the device (device writes data to it and it can read data from card).

Perhaps after initializing the SD card is stuck in SPI mode and can't be brought back into SD mode (though judging from the SD card spec the card should allow this with each power up sequence). If you have an integrated SD Host Controller (e.g. some Intel tables and NUCs have that; a Raspberry Pi might be an idea as well, though you'll have to boot from USB and have a SD card to microSD adapter) it might be possible to check in the log if something goes wrong there.

I believe dseligo has an SD card that lacks a standard FAT table.  it takes 3x the writes to the SD card to properly support FAT32/FAT16 so for both speed and power reasons a lot of firmware just uses something else.
I have seen utilities On both Windoes paid software (winhex?) and Unix free utility (??) that can access the card on a block level, bypassing the OS file manager and the need for a good FAT table on the card.
Can anyone provide better help with this info?

Again, it does not matter if the SD card is formatted with FAT or not. If the whole disk is not available in disk management then something else went wrong.

MarkMLl

  • Hero Member
  • *****
  • Posts: 6676
Re: SD card low level read
« Reply #14 on: June 21, 2022, 11:06:48 am »
No. I have these devices which initialize SD card and after that cards aren't recognized in Windows or Linux at all. Cards work just fine in the device (device writes data to it and it can read data from card).

What do you mean? Do you mean that it isn't auto-mounted, or do you mean that it doesn't appear as e.g. /dev/sdb?

If it doesn't appear as /dev/sdb then what does the kernel log tell you?

In any case, what sort of device are you using between the card and the PC?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018