Musings of a dad with too much time on his hands and not enough to do. Wait. Reverse that.

Tag: powershell (Page 6 of 7)

Music to drive by, Part 2

A while back, I posted about my challenges putting together a flash drive containing the music I’d like to listen to in the car when not listening to podcasts and so forth. Well, I’d like to offer an update on that effort.

In my original post, I discussed a script that first inventoried all my music then copied select mp3s to my flash drive based on whatever criteria I chose–in my example, my primary criteria was the song’s genre. In retrospect, I think it might be smarter to break those operations out into two scripts: one script to inventory my music and write the inventory to a JSON file and a second script to read that file, apply whatever filtering criteria I wish to apply, and copy the resulting mp3 files to a destination like a flash drive.

Create my inventory file

This script allows me to inventory my music and write it to a file. I’ll only need to re-run this script whenever there’s a change to my inventory. Having this file will, of course, allow me to proceed with the next step of this process–writing select files to my flash drive–but it will also help me figure out if I have music incorrectly labeled and it can also serve as a data file on which I can do some analysis later on. Here’s a snippet of the important piece of this script, but the full version is available on my Github page:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$music_folder = "$Env:USERPROFILE\Music"  # path to my music files
$dirs_to_exclude = "$Env:USERPROFILE\Music\soundclips"  # sub-dirs to exclude from the music collection
$mp3_collection = @()

# get a list of all the sub-directories containing MP3 files
$mp3_folders = dir "$music_folder\*.mp3" -Recurse | select Directory -Unique | where {$dirs_to_exclude -notcontains $_.Directory.FullName}

# now, loop through all my music folders to collect all the MP3s I want to process; store them in the $mp3_collection collection object
foreach($mp3_folder in $mp3_folders){
    $shell = (New-Object -ComObject Shell.Application).NameSpace($mp3_folder.Directory.FullName)
    foreach($mp3_object in $shell.Items()){
        if($shell.GetDetailsOf($mp3_object, 2) -like "MP3 File"){
            $mp3_file = [pscustomobject]@{'file' = $mp3_folder.Directory.FullName + '\' + $shell.GetDetailsOf($mp3_object, 0);
                                          'artist' = $shell.GetDetailsOf($mp3_object, 13);
                                          'album' = $shell.GetDetailsOf($mp3_object, 14);
                                          'album_year' = $shell.GetDetailsOf($mp3_object, 15);
                                          'genre' = $shell.GetDetailsOf($mp3_object, 16);
                                          'song' = $shell.GetDetailsOf($mp3_object, 21);
                                          'file_size' = $shell.GetDetailsOf($mp3_object, 1);
                                          'length' = $shell.GetDetailsOf($mp3_object, 27)}
            $mp3_collection += $mp3_file
        }
    }
    $shell = $null
}

# save collection to json object
$mp3_collection | ConvertTo-Json -Depth 5 | Out-File ($music_folder + "\mp3_collection.json")

Here’s an example of the JSON file the script will create:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
[
    {
        "file":  "C:\\Users\\jdoe\\Music\\AC-DC\\Back in Black\\01_AC-DC_Hells Bells.mp3",
        "artist":  "AC/DC",
        "album":  "Back in Black",
        "album_year":  "1980",
        "genre":  "Hard Rock \u0026 Metal",
        "song":  "Hells Bells",
        "file_size":  "7.25 MB",
        "length":  "00:05:11"
    },
    {
        "file":  "C:\\Users\\jdoe\\Music\\38 Special\\The Very Best Of The A\u0026M Years (1977-1988)\\01 - Rockin\u0027 Into The Night.mp3",
        "artist":  "38 Special",
        "album":  "The Very Best Of The A\u0026M Years (1977-1988)",
        "album_year":  "2003",
        "genre":  "Rock",
        "song":  "Rockin\u0027 Into The Night",
        "file_size":  "7.56 MB",
        "length":  "00:03:59"
    },
    {
        "file":  "C:\\Users\\jdoe\\Music\\Billy Idol\\Greatest Hits\\03 - Hot In The City (2001 Digital Remaster).mp3",
        "artist":  "Billy Idol",
        "album":  "Greatest Hits",
        "album_year":  "2001",
        "genre":  "Rock",
        "song":  "Hot In The City (2001 Digital Remaster)",
        "file_size":  "7.16 MB",
        "length":  "00:03:32"
    }
]

Copying select files to my flash drive

My second script in this process will pull in my inventory file, apply whatever filters I’ve coded, and write the results to whatever destination I chose. Furthermore, I’ve added a check to limit the overall size of selected files by the size of the flash drive you’re using. Here’s a snippet of my code, but the full version is available on my Github page:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
function Convert-FileSizeToMB($size){
    $return_val = 0

    switch ( $size.split(" ")[1] ){
        "GB" { $return_val = [double]$size.split(" ")[0] * 1GB }
        "MB" { $return_val = [double]$size.split(" ")[0] * 1MB }
        "KB" { $return_val = [double]$size.split(" ")[0] * 1KB }
    }

    return $return_val
}

$music_folder = "$Env:USERPROFILE\Music"  # path to my music files
$mp3_col = Get-Content ($music_folder + "\mp3_collection.json") | Out-String | ConvertFrom-Json
$flashdrive_location = "C:\temp_music_folder"  # set the location of your flashdrive here
$flashdrive_size = 14250MB  # my song selections will likely exceed the size of the flash drive, so set that size limit here

# song selection criteria
$genres_i_want = "Metal", "Hard Rock & Metal", "Rock", "Rock; Hard Rock & Metal"
$bands_to_skip = "Mel Tormé", "Starland Vocal Band", "Burt Bacharach"

# apply my selection criteria and get a list of the songs to copy over to the flashdrive
$mp3s_to_write_to_drive = $mp3_col | where {$genres_i_want -contains $_.genre} | where {$bands_to_skip -notcontains $_.artist}

$size_of_files = 0
foreach ($mp3 in $mp3s_to_write_to_drive){
    if ($size_of_files -le $flashdrive_size){
        $size_of_files += Convert-FileSizeToMB $mp3.file_size
        Copy-Item -LiteralPath $mp3.file $flashdrive_location
    }else{
        break
    }
}

Unfortunately, this script runs quite long–at least, when I’m trying to load a 16 GB flash drive. In my experience, PowerShell just doesn’t do well with big I/O operations. If this gets too annoying, I may try to throw Python at the problem and see if I can get a better runtime.

Why am I going through all this effort when I should just be able to write my whole catalog to a large flash drive?

So, you might be asking, why are you doing all this hard work when surely your music collection can fit on today’s large capacity flash drives? First of all, don’t call me Shirley (sorry, couldn’t resist). To start with, my car seems to require FAT32 formatted flash drives. Once you get above 32 GB flash drives, though, it becomes difficult, but not impossible, to format such large drives as FAT32. I did this, in fact, some time ago. I took a 64 GB flash drive, formatted it to FAT32, and copied my entire catalog to it. It worked…kind of. Strangely, my car radio only recognized a fraction of all the songs I had on the drive. Then, after a few months, my drive more or less burned up and became unusable. So, I decided to go a less radical route and try to build a more selective process–the one described above.

Posting code frustrations

Either I’m missing something or posting code snippets on WordPress sites is just not easy–or, at least, I haven’t yet stumbled upon the trick of it.  So far, I’ve posted PowerShell and Python code snippets and I’ve used the CodeColorer plugin to jazz up the snippets a little.  Initially, this worked fine with my PowerShell snippets:

1
2
3
$birth_date = [datetime]"3/29/1916"
$death_date = [datetime]"6/23/1984"
"Tom was {0} years old when he died" -f [math]::floor((new-timespan -start $birth_date -end $death_date).Days / 365.2425)

(I’m wrapping my code with the CodeColorer syntax [ccn lang=”powershell” width=”600″] )

However, Python depends heavily on tabbing to denote if statements, for loops, and so forth and this adds another level of complexity to the snippets you post.  In HTML, to properly preserve tabs and spacing in your code, you generally have to turn the pre tag.  So, for my Python snippets, I turned to both the CodeColorer plugin and the SyntaxHighlighter TinyMCE Button to wrap my snippets in appropriately styled “pre” tags and get this:


1
2
3
4
5
6
7
8
#the old surround-the-string-with-parentheses technique
qry1 = ('SELECT fname ' +
        ',lname ' +
        ',job_title ' +
        'FROM people')

curs.execute(qry1)
curs.fetchall()

(Here, I’m wrapping my snippet with this: <pre class=”brush: python; gutter: true; first-line: 1″>[ccn lang=”python” width=”600″])

What in the world is going on with the large gray box around my code?  It appears to be a background style setting connected to the pre tag, complements of the stylesheet my theme uses. Fortunately, WordPress has a custom CSS feature, so I should be able to overwrite this behavior.

It took me a few attempts, but this is custom CSS I came up with: .preforcode pre {background: transparent}

To implement my custom class, I wrapped my “pre” tag with a div, like so (note that I think all that brush…python stuff that the SyntaxHighlighter TinyMCE Button places in the class attribute of the pre tags is nonsense, so I just removed the attribute altogether):
<div class=”preforcode”><pre>[ccn lang=”python” width=”600″]


1
2
3
4
5
6
7
8
#the old surround-the-string-with-parentheses technique
qry1 = ('SELECT fname ' +
        ',lname ' +
        ',job_title ' +
        'FROM people')

curs.execute(qry1)
curs.fetchall()

Sweet! That gray box is gone! The padding around my snippet is still there: I guess that kind of makes my snippet pop out, but I might now have to mess around with that a little and eventually add any modifications I come up with to my custom CSS. At any rate, at least the snippet looks better so, going forward, hopefully my code snippets will look a little more appealing.

Working around Excel

Every-so-often I find myself having to do some processing with a Microsoft Excel file–grouping data, performing different statistical operations, etc. Doing much of this work within Excel itself is just a pain, so I would always open the file and save the applicable worksheets to CSV files.  But even manually saving the worksheets to CSV was a pain, so I devised a way to do the whole thing with PowerShell.

Unfortunately, PowerShell still has no convenient Convert-ExcelToCSV cmdlet…so I wrote my own. For added convenience, I built my function as a module and even added an Import-Module statement in my profile. Here are some quick steps on how I did that:

  1. Under your %userprofile%\documents directory, make sure you have a WindowsPowerShell directory. If not, create one.
  2. Under that WindowsPowerShell directory, create a modules directory, if one doesn’t already exist.
  3. Finally, under the modules directory, create a directory called Convert-ExcelToCSV.
  4. In that Convert-ExcelToCSV directory, drop a copy of my Convert-ExcelToCSV.psm1 module.
  5. Back under the WindowsPowerShell folder, create a file named Microsoft.PowerShell_profile.ps1.
  6. Edit Microsoft.PowerShell_profile.ps1 and add the statement Import-Module Convert-ExcelToCSV

(For more on PowerShell profiles, check out this Scripting Guy post)

So, that’s pretty cool. Now, you can open up a PowerShell command shell and save your Excel files to CSV in a one-liner:

Convert-ExcelToCSV -FileName "C:\some\path\my_excel_file.xlsx"

But we can still make this even more convenient: how about, when I right-click on an Excel file, I get the option to convert the sucker to CSV right there? Well, we can do that by editing the Windows Registry.

According to my research, in order to cover the older XLS file versions as well as the newer XLSX versions, you have to edit two slightly different paths under the HKEY_CLASSES_ROOT hive. Check out my registry file for enabling this feature for the older file type and my other file for enabling this feature on new versions of Excel files.

To make these registry modifications, you can download my REG files and run them on your machine.

Warning: editing your Registry is a big deal. I’ve tested these on my system and they work, but make sure you review the modifications first and make sure you’re comfortable with everything before proceeding.

With these modifications, you can now convert your Excel files to CSVs with ease.

 

Python makes it even easier

So, I had to do a lot of work to get to a point where I can easily convert my Excel files to CSVs for easier processing down the line. Well, guess what? Python makes it even simpler with pandas.  Check out the function read_excel.  That is just awesome!

So, either way you go–PowerShell or Python–you have options to free yourself from some of the frustrations of Excel!

« Older posts Newer posts »

© 2024 DadOverflow.com

Theme by Anders NorenUp ↑