DadOverflow.com

Musings of a dad with too much time on his hands and not enough to do. Wait. Reverse that.

Page 49 of 57

More handy PowerShell snippets

June 13, 2018 /

In another installment of “handy PowerShell snippets“, I offer a few more I’ve used on occasion:

Comparing documents in PowerShell

WinMerge is a great tool for identifying differences between files, but if you want to automate such a process, PowerShell’s Compare-Object is an excellent choice.

Step 1: Load the documents you wish to compare



1
2
$first_doc = cat "c:\somepath\file1.txt"

$second_doc = cat "c:\somepath\file2.txt"

 Step 2: Perform your comparison.
Note that Compare-Object will return a “<=” indicating that a given value was found in the first file but not the second, a “=>” indicating a given value was found in the second file but not the first, or a “==” indicating that a given value was found in both files.



1
$items_in_first_list_not_found_in_second = ( Compare-Object -ReferenceObject $first_doc -DifferenceObject $second_doc | where { $_.SideIndicator -eq "<=" } | % { $_.InputObject } )

 Step 3: Analyze your results and profit!

One note of warning: In my experience, Compare-Object doesn’t do well comparing nulls. To avoid these circumstances, when I import the files I wish to compare, I’ll explicitly remove such troublesome values.



1
$filtered_doc = ( Import-Csv "c:\somepath\somedoc.csv" | where { $null -ne $_.SomeCol } | % { $_.SomeCol } )

 

Join a list of items into a single, comma-delimited line

Sometimes I’ll have a list of items in a file that I’ll need to collapse into a single, delimited line. Here’s a one-liner that will do that:



1
(cat "c:\somepath\somefile.csv") -join ","

 

Use a configuration file with a PowerShell script

A lot of times, PowerShell devs will either declare all their variables at the top of their scripts or in some sort of a custom configuration file that they load into their scripts. Here’s another option: how about leveraging the .NET framework’s configuration system?

If you’ve ever developed a .NET application, you’re already well aware of how to use configuration files. You can actually use that same strategy with PowerShell. For example, suppose you’ve built up a configuration file like so:



1
2
3
4
5
6
<?xml version="1.0" encoding="utf-8" ?>

<configuration>

  <appSettings>

    <add key="test_key" value="dadoverflow.com is awesome and i'm going to tell my friends all about it"/>

  </appSettings>

</configuration>

You can then load that config file into your PowerShell script with the following:



1
2
3
4
$script_path =$MyInvocation.MyCommand.Path



$my_config =[System.Configuration.ConfigurationManager]::OpenExeConfiguration($script_path)

$my_config_val = $my_config.AppSettings.Settings.Item("test_key").Value

One note: your PowerShell script and config file will need to share the same name. If your PowerShell script is called dadoverflow_is_awesome.ps1, then you’ll want to name your config file dadoverflow_is_awesome.ps1.config.

Here’s a bonus: Yes, it might be easier to just declare your variables at the top of your file and forgo the extra work of crafting such a config file. However, what if one of your configuration values is a password? By leveraging .NET’s configuration system you also get the power to encrypt values in your config file and hide them from prying eyes…but that’s a discussion that merits its own blog post, so stay tuned.

Handy PowerShell snippets

June 10, 2018 /

I code a fair amount with PowerShell at work and home and find myself reusing different snippets of code from script-to-script.  Here are a few handy ones I like to keep around.

Get the directory of the executing script

Having the directory of the executing script can be handy to load adjacent resources or as a location to which to write logs or other data:



1
$ExecutionDir = Split-Path $MyInvocation.MyCommand.Path

Dynamically add a new column to a CSV imported into a collection

Many times you need to add one or more columns to a data file you’re working on. Here’s a way to load your data file and add those other columns in one line:



1
Import-Csv "C:\somepath\some.csv" | select *, @{Name='my_new_column'; Expression={'some value'}}

Test whether an object is an object or an array

One thing I find frustrating with PowerShell is that when you retrieve an object, say through a web request or simply filtering on a collection, you don’t necessarily know the datatype of the result set. You could either have an array of objects or a single object. The problem is, the available properties change between arrays and single object. If you try to print “count” on a single object, for example, PowerShell will throw an exception. In order not to crash my scripts, then, I’ll use code like what I have below to test the datatype of my object before continuing on:



1
2
3
4
5
6
7
if($null -ne $myObj){

    if($myObj.GetType().IsArray){

        # $myObj is a collection, so deal with it as such`

    }else{

        # $myObj is a single object

    }

}

Add attributes to a XML document

Manipulating XML documents can be a real pain. Here’s an easy way to add an attribute to a XML element:



1
2
3
$x = [xml]"&lt;top_level_element/&gt;"

$x.DocumentElement.SetAttribute("my_attribute", "some value")

$x.OuterXml

Upload a document to a Sharepoint document library

I suspect there are probably easier ways to do this with the Sharepoint web APIs, but here’s a technique I’ve used in the past to upload a document to a document library in Sharepoint:



1
2
3
4
$web_client = New-Object System.Net.WebClient

$web_client.Credentials = [System.Net.CredentialCache]::DefaultCredentials

$my_file = gci "C:\somepath\somefile.txt"

$web_client.UploadFile( ("http://some_sharepoint_domain/sites/some_site/Shared Documents/{0}" -f $my_file.Name), "PUT", $my_file )

Use the HTML Agility Pack to parse an HTML document

Parsing HTML is the worst! In the Microsoft world, some genius came up with the HTML Agility Pack allowing you to effectively convert your HTML page into XML and then use XPath query techniques to easily find the data you’re interested in:



1
2
3
4
5
6
7
Add-Type -Path "C:\nuget_packages\HtmlAgilityPack.1.8.4\lib\Net40\HtmlAgilityPack.dll"



$hap_web = New-Object HtmlAgilityPack.HtmlWeb

$html_doc = $hap_web.Load("https://finance.yahoo.com/")

$xpath_qry = "//a[contains(@href, 'DJI')]"

$dow_data = $html_doc.DocumentNode.SelectNodes($xpath_qry)

$dow_stmt = ($dow_data.Attributes | ? {$_.Name -eq "aria-label"}).Value

Convert one collection to another (and guarantee column order)

First, imagine you have a collection of complex objects, say, a JSON document with lots of nesting. You want to try to pull out just the relevant data elements and flatten the collection to a simple CSV. This snippet will allow you to iterate through that collection of complex objects and append simplified records into a new collection. Another problem I’ve found is that techniques like Export-Csv don’t always guarantee that the columns in the resulting CSV will be in the same order you added them in your PowerShell script. If order is important, the pscustomobject is the way to go:



1
2
$col2 = @()

$col1 | %{ $col2 += [pscustomobject]@{"column1"=$_.val1; "column2"=$_.val2} }

Load multiple CSV files into one collection

It’s not uncommon to have multiple data files that you need to load into one collection to work on. Here’s a technique I use for that situation:



1
2
3
4
5
$col = @()

dir "C:\somepath" -Filter "somefile*.csv" | % { $col += Import-Csv $_.FullName }



# If you need to filter out certain files, try this:

dir "C:\somepath" -Filter "somefile*.csv" | ? { $_.Name -notmatch "excludeme" } | % { $col += Import-Csv $_.FullName }

Parse a weird date/time format

It’s inevitable that you’ll run into a non-standard date/time format that you’ll have to parse. Here’s a way to handle that:



1
$date = [datetime]::ParseExact( "10/Jun/18", "dd/MMM/yy",$null )

 

Wrapping up the school year

June 6, 2018 /

In the US, this is the time when schools let out for the summer–well, actually, that time began a week or two ago, but just humor me here.  Clearly, this is an important transition time from school year to summer break.  The rigor of the school schedule is gone, the “free” yellow bus transportation services are no more, late night homework and study sessions take a hiatus.  We’re now looking at the blank canvas of Summer.  Of course, as parents, our schedules don’t change and, in many ways, become harder as we struggle to find summer activities for the kids and ways to transport them there.

Nevertheless, as our children wrap up their school years, here are some actions I try to take to make the transition into Summer a little easier and more organized:

Scan, Scan, Scan

Hopefully, you’re scanning the hard copy documents, awards, and pictures your children bring home with them all throughout the year (and otherwise retaining electronic versions of report cards, etc).  Scanning, say, an award certificate as soon as your child receives it can be helpful for at least two reasons: 1) your scanner will affix the current date to your file helping you know, years from now, when your child earned that award (particularly if the award date isn’t on the certificate) and 2) the affixed date will help you organize such files so that it’s easier to include them with other, related family media.

Regardless, your children will still bring home large amounts of documents in the days before school ends.  I try to go through all these documents and scan as many as makes sense.  School photos?  Check.  Awards and certificates?  Absolutely.  Report cards, standardized test results, and the like?  Definitely.  Writing assignments, poems, and so forth?  Yep.  Going through the effort of scanning these items now can be time consuming, but is well worth it years later when the hard copies are long gone and all you have are those electronic memories.

Take Pictures

I certainly take pictures of my children at their various school events throughout the year and especially at their various end-of-year ceremonies.  I also try to take a few pictures of them in their school uniforms and at the bus stop awaiting its arrival.  I try to include their fellow travelers, as well, as some of those students may be graduating and moving on.  One could even go so far as to print up a sign saying something like, “last day for 4th grade!” and have his child hold the sign in a picture–I wouldn’t recommend that for your older children, though.

Consider filing your child’s notes and related documents

My oldest is a fantastic note taker.  On more than one occasion, we’ve been able to pull her notes out of storage and use them to help the younger ones get through a class or two the older one had already traversed.  Here’s another true story: for my first semester of college, I studied abroad in Singapore.  When I returned to campus in January, I learned that students could take placement tests to test out of certain required classes.  So, I went home and collected the two or three three-ring binders I kept of my Calculus notes from my senior year in high school.  I studied these for several days, took the placement test, and was able to test out of three semesters of math classes.  Cheap credit hours there!  Thus, retaining and organizing your child’s class notes and associated documentation could be a worthwhile endeavor.

Say, “Thanks”

“Thank yous” are free–except for the outrageously priced thank-you cards.  At any rate, your children should thank their teachers, school staff, and even the bus driver.  A card and gift might even be appropriate.  A “thank you” now might be worth a recommendation letter later.

Postmortem?

Often at work, after a project, the team will gather to do a “postmortem” or collect “lessons learned”–simply a list of things that went well, things that didn’t, and ways future projects can learn from the one just finishing.  You could consider helping your child do the same.  What did he like about the year?  What did he hate?  What teachers and classes did he enjoy and why?  Which ones did he not enjoy?  What tools did he use to study for tests and organize his assignments?  Did these work out for him or should he spend some time finding replacements?  These moments of introspection might help your child become more efficient next year.  They could help him decide on career and major as college draws near.  They could help you better coach any younger children you have following in this child’s footsteps.  You might even share your notes with parents of other, younger children entering the grade your child just completed.

Plan out the summer

What, you thought you had a break for a couple of months?  To me, summer is a great time to get a jump on the competition, er, fellow students.  I don’t know if it’s good or bad, but schools seem much more academically–and, certainly, athletically–competitive than they did when I was young.  So, while the competition is lounging by the pool, consider enrolling your children in summertime academic courses that can help them get ahead.  In summers past, we’ve enrolled our kids in Mathnasium and different sessions at the local college and community center.  Ask yourself, what are some of your child’s academic goals?

  • Competing to get into a higher math class?
  • Preparing for an upcoming standardized test?
  • Simply easing some of the anticipated pressures of the next school year?

Summertime academic courses can help meet these objectives and more.

Also new these days is the dreaded Summer homework.  Establishing a plan for your child to knock these requirements out early seems a wise thing to do.  Hopefully, you also have some idea of projects to expect in the next school year.  For example, my child will have to construct a science fair project in his next year.  I hope to spend some of this Summer at least brainstorming with him what we might try to tackle and maybe even begin gathering relevant material.  Whatever your plans are, develop and implement them early before your child gets too used to a life of leisure.

Get a teacher recommendation

If your child is already a junior or senior in high school, he either already has one or more recommendation letters or is actively working on getting one for his college applications, assuming he’s going that route.  But what if your child simply needs a recommendation for a job or maybe an internship?  Before the school doors close and the teachers disappear into the hinterlands, get a recommendation letter if you think you might need one.

Friend contact information

When you see your friends everyday at school, you may not think about how to contact them outside of the establishment.  But if your child wishes to hang out with his school chums over the break, he’ll want to make sure to get phone numbers and email addresses of them before the building shutters.

Read a good book

During the summer, kids usually can’t make the claim that they have no time to read.  Challenge your child to read a good book or two: how about one from this list?

Talk to other parents

Before the year ends, I try to chat up a parent or two of students one grade higher than mine.  What sorts of challenges should my kid be aware of next year?  What teachers and classes should we be wary of?  And so on.  Gather some intelligence on the next grade and prepare for those challenges over the break so you’re ready for them.

Sell your books?

Academic books can be outrageously expensive.  If you really think you won’t need them for reference or for a younger sibling, consider selling those books to another student and try to recoup some of your expense.

 

Any other tips I missed?  If so, please leave them in the comments!

« Older posts Newer posts »

© 2025 DadOverflow.com

Theme by Anders Noren — Up ↑