Today we unearthed yet another old CD-ROM. We used to backup all our family photos and videos onto CDs and DVDs in the good old days of slow ADSL connections and expensive or non-existent cloud storage.
These days, when I come across these CDs and DVDs, I use this quick hack to send them into the cloud.
Step 1
Pop them into a DVD or CD reader. I have one attached to my trusty Fedora laptop.
Step 2
Determine the Linux device file for the DVD/CD reader.
In my case, the CD/DVD auto-mounts. So a simple ‘df ‘ gives the device name /dev/sr0 – highlighted in the transcript below.
Use ‘dd’ to copy the CD/DVD’s contents onto a location on your computer.
‘dd’ is part of the GNU Coreutils package and should be installed by default on every Linux instance.
$ dd if=/dev/sr0 of=/home/user1/PhotoCD1.img bs=2048
282424+0 records in
282424+0 records out
578404352 bytes (578 MB, 552 MiB) copied, 237.454 s, 2.4 MB/s
$
Note that specifying the block size of the transfer using ‘bs=2048’ is optional. It simply refers to the block size you want to use to buffer your I/O transfer. The operating system should auto-select this. I chose to do it out of (old-school) habit because /dev/sr0 was reporting a 2048-byte optimal I/O size (see highlighted lines):
Do with the resultant ‘virtual’ CD/DVD (/home/user1/PhotoCD1.img in the example above) as you will.
I usually confirm first that it is what I think it is:
$ file /home/user1/PhotoCD1.img
/home/user1/PhotoCD1.img: ISO 9660 CD-ROM filesystem data 'PhotoCD1'
$
Then I test mounting it to and navigating to it on my laptop (double-clicking in your default file manager program should do the trick, then examine its contents).
Finally I copy it to my chosen online backup storage.
I am coming out of hibernation briefly to wish you all the very best for 2023! May it be filled with peace, hope and success.
The last three months have seen me adding study (a Graduate Certificate in an I.T. discipline), to the existing juggle of family and work. Blogging has been the inevitable casualty which has been disappointing for me personally, even though I am super-excited that I get to dabble in I.T. again! This blog is still live, but updates will be infrequent until I finish study, hopefully early 2024.
Earlier this year I got to visit Nkhotakota Pottery Lodge where visitors can create their own pottery pieces. This post’s featured image is my attempt to paint Girl Tux onto pre-baked pottery tiles. It was a fun exercise, but also gave me an appreciation of how difficult it is to produce a beautiful piece of pottery!
Today I am interviewing my daughter, who will be sitting for her final senior high school exams shortly. While this interview isn’t related to the subject of diversity in tech, it comes close: she is taking Science and Math subjects which are part of STEM (Science, Technology, Engineering & Mathematics), and hopes to study Engineering at university next year.I thought it would be interesting to hear some of her perspective.
Note that my daughter’s responses below relate to her experiences (and hers only) in two Australian high schools that she has attended. Her references to year levels follow the Australian education system.
Question: Why did you choose to study STEM subjects in high school?
Answer: I guess there are three main reasons. I enjoy STEM subjects, and they make sense to me. As in, I find there is a rule to everything. Second, for STEM subjects, I don’t have to write essays based on opinion, or essays that seek to persuade. Finally, I need to take these subjects in order to be accepted into the engineering university course I’ve chosen.
Question: Do you know many other girls who are studying STEM subjects in the high schools you’ve attended?
Answer: There’s been a balance: in my previous high school [her first three years of high school] several girls in my friendship group were studying both STEM and Humanities/Arts subjects. I don’t know how many of them kept studying STEM subjects in their senior school years. In my current high school, 75% of students studying Chemistry and 50% of students studying Physics are female.
Question: So, in your opinion and based on your observations, is there a gender disparity in high school students studying STEM subjects?
Answer: I think there is in Maths and Engineering. Funnily enough in my current school [final two years of high school] there are two girls and one boy in my year level doing Specialist Maths.* But in my previous school there were only boys doing Specialist Maths in Years 11 & 12 [final two years of high school]. And the majority of students doing Engineering as a subject in my previous high school were boys. From what I saw in my previous high school, more boys were interested in doing subjects that involved coding and programming compared to girls. Including myself.
Question: What do you see yourself doing 10 years from now?
Answer: i want a career in Science or Engineering. I just don’t know what that would look like yet. I’d like to be in a job that helps people in some way, even if it is behind the scenes – for example, in a lab or in a desk job.
That’s all! Note that comments have been disabled on this post since it relates to a minor.
I use qrencode to generate Quick Response (QR) codes. While there are many web-based and GUI methods of generating these, I decided that I wanted an open source and command-line based method. qrencode is the easiest one to use that I’ve found – so far.
qrencode uses the libqrencode library to produce a QR Code symbol.
This generates the following QR code image and saves it to output file test.png:
Test QR Code generated by qrencode on Linux
Note that:
The ‘-d’ specifies the resolution (in dpi or dots per inch) of the resulting output image.
The ‘-o’ specifies the output file to write the resulting QR Code to.
PNG is the default output image type anyway, so I didn’t need to specify the “.png” in the output file name above. If I wanted to change the filetype, I can do so by specifying a ‘-t’ or ‘–type’ followed by the file type I want.
You can check out all qrencode’s options either in its documentation found here, or using the Linux man pages by typing:
$ man qrencode
on the Linux command-line.
Linux Magazine has some more information on qrencode which you can find here.
In Australia most major universities have been recently offering fast-track courses in Cyber Security and Data Science in an effort to bridge this gap. I’ve been checking them out as I’m on a mission to keep my ICT skills current.
I’m working, handling home duties, and supervising two high-school students’ distance learning (this is because of the remote location we live in). I have done the math, and know I can manage an 8-10 hour study commitment per week. I also know from past – but recent – experience that 8-10 hours is generally the commitment required, per subject, for an Australian undergraduate semester-based university course. So I headed happily to Aussie university websites to research course offerings.
My personal criteria were:
Postgraduate coursework in either Artificial Intelligence (or related area) or Cyber Security.
Delivered entirely online.
Allowing one subject to be studied at a time.
8-10 hour commitment per subject per week.
It wasn’t an easy as I thought. Most courses that met the first three criteria failed to meet the fourth. Many turned out to be fast-track courses in Data Science or Cyber Security with a minimum 20 – 24 hour commitment per week, albeit for a fewer number of weeks per year compared to traditional semester based courses.
A 20-24 hours per week study commitment is simply impossible for me. To put it in context, someone working full-time in the Australian ICT sector would be expected to commit to 38 – 40 work hours per week. A 20 – 24 hour commitment per week is effectively half-time – a significant commitment.
I tried to negotiate a lighter study load with two universities. This was unsuccessful. In fact one student advisor assured me that the fast-track post-graduate course under discussion had several students who were successfully working full-time, AND engaging with the coursework. I felt a bit of a dunce as I was clearly incapable of handling life the way my peers were able to.
Once I got off the phone though, I started to think. What demographics did these students par excellence fall into? Was their employer giving them time off in order to study? (My previous employer would frequently do so for employees wanting to upskill in the industry.) Were these students actively parenting? Or were they parenting as a couple? That is, they had partners handling parenting and home-duties so these students were freed up to study?
Note that in my hunting I did discover some courses with the 8-10 per subject per week commitment – but only for on-campus enrollments. I.e, they didn’t meet my personal criterion # 2 above.
Confession: I felt annoyed. Here were very reputable higher-education institutions, offering important ICT skills training, that were difficult for people like me to access. People who are juggling work, parenting and home-duties. People like me who are (almost inevitably) female, mid-career and looking to upskill or reskill.
But here’s the rub. We talk a lot about attracting women into the ICT sector. But if we truly want women in ICT, it can’t just stop at attracting young women into the industry. It also has to be about keeping women in ICT by encouraging ongoing skill development. This means resourcing women where they are at – whether in or out of work, whether as singles or not, whether parenting singly or jointly, whether relatively free of responsibility or caring for dependents, whether managing a business or as an employee, whether as new entrants into the industry or mid-career professionals or as those close to retirement. You could probably add to this list.
This isn’t just a bad news story though. I kept on hunting, and thankfully found two Australian universities who offer a course meeting all four of my criteria above. I’m applying to study next year, and hoping I will be accepted!
So, what do you think? I’ve written here about the Australian scene, because that’s what I’m familiar with. Please feel free to leave a comment about what you have experienced in this and other geographic locations.
I recently saw some requests for help with LVM for Linux on sysadmin forums. In my opinion, there were some clear gaps in the posters’ understanding of LVM architecture. So today I’ll give an overview of LVM’s basic components.
What is LVM?
First of all LVM stands for Logical Volume Manager. Chances are that your distro has it pre-installed – but if it not, LVM is reasonably straight-forward to install – refer to your distro’s documentation.
LVM is a toolset that we can use to put a bunch of storage devices into a group, and carve up the contents of that group into usable portions. Generally, we would then go on to create filesystems on those portions, or use them for swap (virtual memory).
The LVM implementation in Linux is related to the LVM implementation in HP-UX [1]. LVM is being used less and less these days with the availability of on-demand, scalable, ‘elastic’ cloud-based storage. However, at this point in time, there are still many home and enterprise Linux installations using LVM.
LVM has a bunch of powerful features, and you can do quite a lot with it. Today, we’re looking at its basic components only.
Physical Volumes (PVs)
At their very basic, Physical Volumes are storage devices attached or presented to a computer. These days in the enterprise computing world, these devices are likely to be virtual devices, but they look and feel like physical devices to the operating system. Before a disk can be used by LVM, it has to be initialised as a Physical Volume (PV). If you’re using the command-line, this is done using pvcreate. This process writes a label on the disk and initialises areas for LVM metadata.
A common misconception is that a disk device can be added directly to a Volume Group without initialising it as a PV first. Not so!
Volume Groups (VGs)
A Volume Group is a collection of Physical Volumes which is then divided up into Logical Volumes. Essentially VGs are a way of grouping together storage devices, so usable portions of storage can be carved out of them. If using the command line, Physical Volumes can be added to a new Volume Group using vgcreate, or added to an existing Volume Group using vgextend.
Logical Volumes (LVs)
A Logical Volume is a portion carved up out of a Volume Group that can be used by the operating system for data in some way. Typically, a Logical Volume will be used to create a filesystem. But not always. For example, a Logical Volume can be used as a swap device (for virtual memory).
From the command line, a Logical Volume can be created in a Volume Group using lvcreate, and have additional storage space added to it using lvextend. Both lvcreate and lvextend have options to specify which PV (or PVs) you wish to use.
Many types of Logical Volumes can be created. If LVM is installed, the help or man pages have a rundown on all the different kinds of LVs possible and the options that can be used with them.
Overview summary
Here are the ‘layers’ involved in a typical basic LVM implementation:
Overview of LVM’s ‘layers’
Note that:
LVM doesn’t encapsulate existing devices or their data. So once a device (disk) has been initialized by LVM as a Physical Volume, any existing data on it is wiped.
A system (server or your personal computer) can have more than one Volume Group. My example above was very basic, showing a system with a single VG.
Just for the record, a Logical Volume can’t span Volume Groups.
Logical Volumes can be grown online as long as there is space available in the Volume Group. Once an LV is grown though, a separate step would be required for the filesystem it contains to make use of the new space in the LV. Documentation on the filesystem’s type will state whether the filesystem can be grown or resized, and whether this is possible to be done online.
Existing Volume Groups can be imported or exported to or from the operating system of a computer. This comes in handy when the storage in a Volume Group needs to be detached from one computer and moved to another.
High-availability add-on options are available that allow Volume Groups to be shared between clustered servers.
Summary of the process
The diagram below shows the steps for creating a basic Volume Group and Logical Volumes.
Creating PVs, VGs and LVs
If you couldn’t see the image above, the steps are reproduced below. First, ensure storage is presented to and visible from the operating system. Then:
Create and initialise physical volume(s).
Create volume group.
Create logical volume(s)
Finally prepare logical volume(s) for use. For example, by creating filesystem(s) or adding as swap volume(s).
To find out more
Documentation that comes (or should come) with your distro – generally accessed using ‘<command> –help’ or ‘man <command>’.
I like Duplicity for backing up data on my home Linux kit because it meets certain criteria. Bear in mind that these may not be important to you!
Duplicity is open source.
It can be used on the command line, from the Linux shell. But for those that prefer it, there are GUI front-end options (eg, Deja Dup for GNOME).
It can take full and incremental backups.
Backup files are stored in tar gzip format. This was a plus for me. I wanted a format that could be relatively easily read by basic Linux tools that are always likely to be at hand.
Duplicity can integrate with cloud storage – eg, S3 on AWS.
Reasons I wouldn’t use Duplicity for critical systems and data:
It’s support for extended file attributes (xattr) like SELinux file contexts is limited. For those who’re interested, some discussions on this are below, but basically Duplicity appears to use an implementation of tar that isn’t capable of backing up xattr.
I came up against this Duplicity limitation when restoring my laptop’s /home filesystem one day, losing SELinux contexts in the process. Thankfully I didn’t have any non-standard SELinux file contexts in place, so a simple ‘restorecon -Rv /home‘ got things back to normal.
I think it’s too risky to use Duplicity for backing up critical systems and data until it has robust support for extended file attributes and SELinux file contexts.
Installation on Fedora
Available from the default Fedora software repositories. Basically, install using:
Here are some examples of using Duplicity on Fedora 34.
Dry-run to an external drive
This example was run as the root user (or it could be run using sudo) because the folders being used for this dry run were owned by different user accounts.
# duplicity --dry-run --name mylaptop_home /home file:///media/1tbusb/backups/mylaptop/home-backups/
Local and Remote metadata are synchronised, no sync needed.
Last full backup date: none
GnuPG passphrase for decryption:
Cannot use empty passphrase with symmetric encryption! Please try again.
GnuPG passphrase for decryption:
Retype passphrase for decryption to confirm:
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1627774508.08 (Sun Aug 1 01:35:08 2021)
EndTime 1627774539.34 (Sun Aug 1 01:35:39 2021)
ElapsedTime 31.26 (31.26 seconds)
SourceFiles 301977
SourceFileSize 88829478341 (82.7 GB)
NewFiles 160403
NewFileSize 88829478341 (82.7 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 160403
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 0 (0 bytes)
Errors 0
-------------------------------------------------
#
I left my user errors in the transcript above, as it’s helpful to see how Duplicity responds in some error scenarios. In this case, me entering an empty passphrase when symmetric encryption will be used for the resulting backup files.
It all looks good, let’s proceed with the backup.
Actual backup to external drive
Again, as the root user (or via sudo):
# duplicity --name mylaptop_home /home file:///media/1tbusb/backups/mylaptop/home-backups/
Local and Remote metadata are synchronised, no sync needed.
Last full backup date: none
GnuPG passphrase for decryption:
Retype passphrase for decryption to confirm:
First and second passphrases do not match! Please try again.
GnuPG passphrase for decryption:
Retype passphrase for decryption to confirm:
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1627774633.42 (Sun Aug 1 01:37:13 2021)
EndTime 1627778822.78 (Sun Aug 1 02:47:02 2021)
ElapsedTime 4189.36 (1 hour 9 minutes 49.36 seconds)
SourceFiles 160401
SourceFileSize 88829623775 (82.7 GB)
NewFiles 160401
NewFileSize 88829623775 (82.7 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 160401
RawDeltaSize 88815026613 (82.7 GB)
TotalDestinationSizeChange 78745337285 (73.3 GB)
Errors 0
-------------------------------------------------
#
Again my errors are left in. In this case, the second passphrase I entered didn’t match the first one, so I was prompted to re-enter it.
Warning! Make sure you record the passphrase in a safe secure place. If you lose the passphrase, you will not be able to decrypt the backup files to read back data or restore!!
Running an incremental backup
To only backup content that has changed since the last full backup, simply re-run the command exactly as we did above.
Duplicity detects the presence of a full backup at the specified target location, and performs an incremental backup instead.
To list what’s been backed up
To list the status of backup sets at the target backup location:
# duplicity collection-status file:///media/1tbusb/backups/mylaptop/home-backups/
Last full backup date: Sun Aug 1 01:36:59 2021
Collection Status
-----------------
Connecting with backend: BackendWrapper
Archive dir: /root/.cache/duplicity/ca2e105abb0be7fd8378849b99843152
Found 0 secondary backup chains.
Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Sun Aug 1 01:36:59 2021
Chain end time: Tue Sep 14 15:07:02 2021
Number of contained backup sets: 2
Total number of contained volumes: 461
Type of backup set: Time: Num volumes:
Full Sun Aug 1 01:36:59 2021 376
Incremental Tue Sep 14 15:07:02 2021 85
-------------------------
No orphaned or incomplete backup sets found.
#
Duplicity reports finding two backup sets at that location, one full, one incremental. The dates of the backups are listed (and yes, it took me a while to actually write and publish this post!).
Testing a restore
A backup is only as good as its ability to be restored. So it’s a healthy practice to test your backups by restoring from them periodically.
I tested the backup above by doing a full restore to a location different to the original files that were backed up:
I was prompted to enter the same GnuPG passphrase that I’d used when backing up the data. Once that was done, Duplicity went ahead and restored the data. Apart from the issue I encountered with losing SELinux file contexts, the restore was successful.
Learn more about Duplicity
As always, this only scratches the surface of what duplicity can do. To find out more:
$ man duplicity
or
$ duplicity --help
or track down the official Duplicity documentation for your favourite Linux distro.
And before using Duplicity, make sure you’re aware of what it can and cannot offer you. As an example, consider its limited capability with extended file attributes, and how this could affect you in the event of data loss.
Remember:
A backup is only as good as its ability to be restored.
(I don’t think I’m the first person to say that!)
Feel free to share your thoughts below. What backup method do you use? Do you think you’ll use Duplicity? Why or why not?
When was the last time you had to play a DVD on your Linux desktop? With many streaming services these days, the need to directly use any physical form of media is becoming less frequent. And I forget how complicated it is to play a DRM DVD on Linux!
Today we started binge-watching an old series (Season 1 of ‘White Collar’, 2009). Friends lent it to us – on DVD! As an aside, I remember the days when we would head to the local video store and rent a bunch of DVDs for the weekend. And the frustration of dealing with the inevitable discs in the pack that would be scratched or damaged. I’m fully a fan of streaming services!
I popped the disc into the tray of our only device with a DVD reader – a HP Notebook circa 2016 and now running Fedora 34. It has VLC Player installed (from the good folks at the VideoLAN project), but that by itself doesn’t have the capability to play a DVD that is encrypted by a Digital Rights Management system (DRM). VLC Player sees the disc, allows you to hit Play… and then does nothing.
What is libdvdcss?
Enter libdvdcss, another project from VideoLAN. It’s a library that can access DRM-managed DVDs without the need for decryption. You can read all about that project at www.videolan.org/developers/libdvdcss.html
libdvdcss isn’t available in the official Fedora software repositories. You will need to enable a third-party repository from RPM Fusion to install it on Fedora. It’s FOSS has an excellent set of instructions that worked on my Fedora 34 – please head over there, as it’s really comprehensive.
Important: Enable RPM Fusion’s tainted free repo at your own discretion, as it contains packages whose usage may be restricted in some geographical locations. Please read RPM Fusion‘s documentation for more information.
Sample libdvdcss installation
Here is the transcript of my attempt at installing libdvdcss on Fedora 34.
$ sudo dnf install rpmfusion-free-release-tainted
Last metadata expiration check: 0:01:24 ago on Sat 16 Jul 2022 19:39:43.
Dependencies resolved.
========================================================================================================================================================================
Package Architecture Version Repository Size
========================================================================================================================================================================
Installing:
rpmfusion-free-release-tainted noarch 34-1 rpmfusion-free 8.6 k
Transaction Summary
========================================================================================================================================================================
Install 1 Package
Total download size: 8.6 k
Installed size: 1.2 k
Is this ok [y/N]: y
Downloading Packages:
rpmfusion-free-release-tainted-34-1.noarch.rpm 7.8 kB/s | 8.6 kB 00:01
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 1.4 kB/s | 8.6 kB 00:06
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : rpmfusion-free-release-tainted-34-1.noarch 1/1
Verifying : rpmfusion-free-release-tainted-34-1.noarch 1/1
Installed:
rpmfusion-free-release-tainted-34-1.noarch
Complete!
$ sudo dnf install libdvdcss
RPM Fusion for Fedora 34 - Free tainted 5.8 kB/s | 28 kB 00:04
Last metadata expiration check: 0:00:01 ago on Sat 16 Jul 2022 19:41:57.
Dependencies resolved.
========================================================================================================================================================================
Package Architecture Version Repository Size
========================================================================================================================================================================
Installing:
libdvdcss x86_64 1.4.3-1.fc34 rpmfusion-free-tainted 73 k
Transaction Summary
========================================================================================================================================================================
Install 1 Package
Total download size: 73 k
Installed size: 178 k
Is this ok [y/N]: y
Downloading Packages:
libdvdcss-1.4.3-1.fc34.x86_64.rpm 24 kB/s | 73 kB 00:03
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 17 kB/s | 73 kB 00:04
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : libdvdcss-1.4.3-1.fc34.x86_64 1/1
Running scriptlet: libdvdcss-1.4.3-1.fc34.x86_64 1/1
Verifying : libdvdcss-1.4.3-1.fc34.x86_64 1/1
Installed:
libdvdcss-1.4.3-1.fc34.x86_64
Complete!
$
And that’s it! VLC Player can now play the DVD without any issues.
geek girl, linux land 