Difference between revisions of "USB Device Diagnostics in Linux"

From Free Knowledge Base- The DUCK Project: information for everyone
Jump to: navigation, search
m
m (→‎udevadm)
 
(8 intermediate revisions by one user not shown)
Line 68: Line 68:
 
  udevadm monitor --subsystem-match=usb --property
 
  udevadm monitor --subsystem-match=usb --property
 
  udevadm monitor --subsystem-match=usb --property --udev
 
  udevadm monitor --subsystem-match=usb --property --udev
 +
 +
udevadm real-time monitoring of usb ports, To see if a device is plugged into USB or unplugged, real time monitoring showing when a USB device is connected or disconnected.
 +
udevadm monitor --subsystem-match=usb
 +
 +
Does not require sudo.  As soon as you connect any USB device, be it a USB flash drive or USB keyboard, mouse, etc details will be displayed. Also when disconnected.
 +
 +
=== dstat ===
 +
This is useful for specifically looking at flash media connected to a USB port.  Sometimes I am trying to write data to a flash drive and nothing seems to be happening.
 +
 +
dstat - versatile tool for generating system resource statistics.  I can look at where my USB drive is mounted and monitor activity with dstat.  Say, for example, my USB flash drive is at /dev/sda2
 +
dstat -D /dev/sda2
 +
will show i/o activity there.  The -d and -D are both ways to show disk statistics.
  
 
=== discover package ===
 
=== discover package ===
Line 125: Line 137:
 
== Has my USB Flash Drive bit the dust? ==
 
== Has my USB Flash Drive bit the dust? ==
 
{{:Has_my_USB_Flash_Drive_bit_the_dust}}
 
{{:Has_my_USB_Flash_Drive_bit_the_dust}}
 +
 +
== Large File Copy to USB Never Completes ==
 +
It is a long standing issue, and we will call it a BUG in the Linux Kernel.  Copying a large file (over 1 GB), such as a digital movie, seems to be problematic when using a destination on an something like USB attached storage.
 +
 +
'''Linux Sucks at Copying Large Files to USB Drives... It just does... I mean... come'on guys can't you fix this?'''
 +
 +
"''When i try to copy something to USB stick, with FAT, it stops near the end, sometimes at 100%. And of course, when i transfer the memory stick somewhere else, it doesn’t contain complete file.'' "
 +
 +
It is best described by an anonymous person on a [broken link] mailing list:
 +
 +
The problem is due to how the linux kernel works and the default settings in some major distributions. The file copy tool is reading a chunk of the source file then writing it to the destination, and updates the progress bar each time it finishes a write. However, to speed things up, Linux takes the data that is meant to be written and immediately tells the program that is has been written, while doing the work in the background. Linux allows some percentage of system memory to be used for this purpose, which with the size of memory these days is usually more than the entire file, so a program can think it has written the whole thing when in reality the copy has just begun. But, when a program tries to close the file, Linux makes it wait for the operation to complete. (else the program might try doing something with a half-written file)
 +
 +
You see this the most when writing large files to slow devices, like USB, but it can show up in other situations too and make it look like the computer is locking up.
 +
 +
The thing I do to “fix” the problem is tell Linux to buffer less data.
 +
 +
echo vm.dirty_bytes=15000000 | sudo tee -a /etc/sysctl.conf
 +
 +
Restart the system.
 +
 +
Another option:
 +
 +
sudo vi /etc/sysctl.conf
 +
add
 +
vm.dirty_bytes=15000000
 +
 +
Restart the system.
 +
 +
The above configuration forces the kernel to flush data to the disk after every 15 MB, so caja can update its progress bar state.
  
 
== Related ==
 
== Related ==
Line 133: Line 174:
 
* [[Ubuntu Installation from a Flash Drive]].
 
* [[Ubuntu Installation from a Flash Drive]].
 
* [[USB Device Diagnostics in Linux]]
 
* [[USB Device Diagnostics in Linux]]
 +
* [[NTFS or exFAT on Flash Drive]] talks about exFAT, NTFS, FAT32 compatibility.
  
  
 
[[Category:Computer_Technology]]
 
[[Category:Computer_Technology]]
 
[[Category:Linux]]
 
[[Category:Linux]]

Latest revision as of 14:09, 13 September 2022

To detect your USB device, in a terminal, you can try: 

lsusb
lsusb -v

The second, verbose, offering more detail.

Device are mainly identified using a pair of hexadecimal numbers, like 04b3:3108

The 4 first hexadecimal digits are the Vendor ID (04b3 = IBM).

The 4 last hexadecimal digits are the Device ID (3108 = ThinkPad 800dpi Optical Travel Mouse).

When your device is listed as "unknown" you can update your local usb-id definition by running update-usbids as root.

Try this: 

lsusb -v | grep -E '\<(Bus|iProduct|bDeviceClass|bDeviceProtocol)' 2>/dev/null

Another way is to dump the information from the running kernel 

sudo cat /sys/kernel/debug/usb/devices

Parse it a little 

sudo cat /sys/kernel/debug/usb/devices | grep -E "^([TSPD]:.*|)$"

USB Input Devices

Input devices include keyboards, mice, remotes, etc.

They do not include flash drives.

xev - Tool that can provide keycodes from input devices. Useful after you have determined your usb input device works and if it is a hid device like a keyboard or mouse. see Multimedia Keys for details on getting the codes from those extra keys in a keyboard.

summary

Another tool that provides extensive details for USB diagnostics: 

sudo lsinput

You may have to install it first, it is part of input-utils 

sudo apt install input-utils

A tool to show the device being recognized when inserted is 

udevadm monitor --udev

The udevadm command cannot be run by itself. You must include a parameter to tell it what to do.

input-utils package

input-utils: utilities for the input layer of the Linux kernel: This is a collection of utilities which are useful when working with the input layer of the Linux kernel (version 2.6 and later). Included are utilities to list the input devices known to the kernel, show the input events that are received by a device, and query or modify keyboard maps. Specifically the tools deal with /dev/input/event* input devices.

  • lsinput - dumps out all the input devices and the associated details
  • input-kbd - dump out the keyboard mapping of a particular event device, must specify the device number such as 'sudo input-kbd 3'
  • input-events - observe input events for watching a specific input device, must specify the device number such as 'sudo input-events 3'

udevadm

udevadm can be used to monitor USB connections 

udevadm -h
Usage: udevadm [--help] [--version] [--debug] COMMAND [COMMAND OPTIONS]
  info         query sysfs or the udev database
  trigger      request events from the kernel
  settle       wait for the event queue to finish
  control      control the udev daemon
  monitor      listen to kernel and udev events
  hwdb         maintain the hardware database index
  test         test an event run
  test-builtin test a built-in command

To monitor USB connections here are some command usage examples: 

udevadm monitor --subsystem-match=usb --property
udevadm monitor --subsystem-match=usb --property --udev

udevadm real-time monitoring of usb ports, To see if a device is plugged into USB or unplugged, real time monitoring showing when a USB device is connected or disconnected. 

udevadm monitor --subsystem-match=usb

Does not require sudo. As soon as you connect any USB device, be it a USB flash drive or USB keyboard, mouse, etc details will be displayed. Also when disconnected.

dstat

This is useful for specifically looking at flash media connected to a USB port. Sometimes I am trying to write data to a flash drive and nothing seems to be happening.

dstat - versatile tool for generating system resource statistics. I can look at where my USB drive is mounted and monitor activity with dstat. Say, for example, my USB flash drive is at /dev/sda2 

dstat -D /dev/sda2

will show i/o activity there. The -d and -D are both ways to show disk statistics.

discover package

use the command discover 

discover

a working example

I have connected a tiny USB receiver for an unknown pointing device. The manufacturer is unknown. I'm not sure if it is the correct dongle and if the device works. Here is where I start: 

udevadm monitor --udev

Now that it is running, I will insert the USB receiver.

A lot of information showed up. Too many lines to share here. Whats important is this one: 

UDEV  [42556.827151] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.7/2-1.7:1.0/0003:26E3:106C.0015 (hid)

I can find the device identification which is: 0003:26E3:106C.0015 and that it is a human interface device (hid) which includes the category of mouse and keyboard

Lets look at 0003:26E3:106C.0015, right now we care about the second and third part of the HEX values each part separated by a :

  • 26E3 = the vendor or manufacturer, as a hex value 0x26e3
  • 106C = the product name, as a hex value 0x106c

Now we have a way to match up the device, and we know the device was recognized by the system.

I use control-c to get out of udevadm monitor and now run 'lsusb' and I get: 

Bus 002 Device 008: ID 26e3:106c  
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
[...]

Now I can see that the device 008 is my device because I match the 26e3:106c up. However, lsusb isn't telling me a name. Must not be in its database.

Lets try the command 'sudo lsinput' which spits out a lot of devices. Look through the output and find the group with the matching hex value. Like this: 

/dev/input/event7
   bustype : BUS_USB
   vendor  : 0x26e3
   product : 0x106c
   version : 257
   name    : "EXCEL EXCELDIGI Wireless Device"
   phys    : "usb-0000:00:1d.0-1.7/input1"
   uniq    : ""
   bits ev : EV_SYN EV_KEY EV_REL EV_MSC

Ok, now I know

  • 26E3 (0x26e3) = EXCEL EXCELDIGI
  • 106C (0x106c) = Wireless Device

I can also tell it is an input device. Next lets see if it provides input to the computer, or more accurately, if the computer is seeing input from the device when I turn the device on and press buttons or move it.

We're going to use the command 'sudo input-events' yet we need to know the device number so we can add it to the end of the command. Above we see from the output of lsinput "event7" so the device number is 7 and we will issue the command like this: 'sudo input-events 7'

Nothing happens until the device is manipulated, such as pressing a button or moving it. Now information begins scrolling up the screen indicating the kernel is getting input from the device. here is a sample: 

19:48:45.979478: EV_SYN code=0 value=0
19:48:45.987504: EV_REL REL_X 13
19:48:45.987504: EV_REL REL_Y -11

Those are input coordinates from pointer events, so now we can tell we have a pointing device.

This concludes the example using the mystery device.

Has my USB Flash Drive bit the dust?

Discussion on testing to see if a USB stick has gone bad - there is no S.M.A.R.T. data to go on when trying to determine if your USB memory stick bit the dust. We will discuss some ways to test the USB flash memory drive here.

First DETERMINE where you USB Flash is mounted so you don't wipe out your primary drive!!!

I simply use the command "fidsk -l|grep sd.." before I insert the flash and again after. It wont always be the same depending on the system and what other media is connected but as an example my flash is on /dev/sdc

badblocks

This is a data destructive test. Don't do this if you have any plans to attempt data recovery on the drive. This test will help you determine if the USB flash drive can still be used or if it needs to go in the rubbish bin.

  1. determine where the USB flash is mounted (for this example sdc)
  2. run the command: 
sudo badblocks -w -s -o usbstick.log /dev/sdc

Notice there is no partition number, just the device sdc (not sdc1 or sdc2 etc...)

The badblocks command here will destroy all data on the flash drive. Failures may be due to the flash drive memory controller or capacity has been maxed on space to remap failed blocks which is another way of saying "too many failed blocks." Either of these issues indicates the drive is toast.

The driver descriptor says the physical block size is 2048 bytes, but Linux says it is 512 bytes

A Parted / GParted error message that is displayed when a failing USB flash drive has been inserted.

GParted is showing a 16MB flash drive is something odd like 58.27GB. Errors pop up when GParted is opened complaining about the driver descriptor.

This could indicate either that (A) your flash drive has a bad memory controller or other physical problem and is toast or (B) it is physically fine however someone used a low-level device tool and mistakenly wrote the incorrect block size onto the drive. You can check if the drive is bad after you attempt to correct the block size. (my examples use sdc but your assignment might be different!) 

sudo dd if=/dev/zero of=/dev/sdc bs=2048 count=32 && sync

This will clear the Master Boot Record.

Result on a 16GB drive: 

# fdisk -l /dev/sdc
Disk /dev/sdc: 14.6 GiB, 15640600576 bytes, 30548048 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Large File Copy to USB Never Completes

It is a long standing issue, and we will call it a BUG in the Linux Kernel. Copying a large file (over 1 GB), such as a digital movie, seems to be problematic when using a destination on an something like USB attached storage.

Linux Sucks at Copying Large Files to USB Drives... It just does... I mean... come'on guys can't you fix this?

"When i try to copy something to USB stick, with FAT, it stops near the end, sometimes at 100%. And of course, when i transfer the memory stick somewhere else, it doesn’t contain complete file. "

It is best described by an anonymous person on a [broken link] mailing list:

The problem is due to how the linux kernel works and the default settings in some major distributions. The file copy tool is reading a chunk of the source file then writing it to the destination, and updates the progress bar each time it finishes a write. However, to speed things up, Linux takes the data that is meant to be written and immediately tells the program that is has been written, while doing the work in the background. Linux allows some percentage of system memory to be used for this purpose, which with the size of memory these days is usually more than the entire file, so a program can think it has written the whole thing when in reality the copy has just begun. But, when a program tries to close the file, Linux makes it wait for the operation to complete. (else the program might try doing something with a half-written file)

You see this the most when writing large files to slow devices, like USB, but it can show up in other situations too and make it look like the computer is locking up.

The thing I do to “fix” the problem is tell Linux to buffer less data. 

echo vm.dirty_bytes=15000000 | sudo tee -a /etc/sysctl.conf

Restart the system.

Another option: 

sudo vi /etc/sysctl.conf

add 

vm.dirty_bytes=15000000

Restart the system.

The above configuration forces the kernel to flush data to the disk after every 15 MB, so caja can update its progress bar state.

Related

Retrieved from "http://wiki.robotz.com/index.php?title=USB_Device_Diagnostics_in_Linux&oldid=20793"