Musings of a dad with too much time on his hands and not enough to do. Wait. Reverse that.

Month: June 2018 (Page 3 of 3)

Handy PowerShell snippets

I code a fair amount with PowerShell at work and home and find myself reusing different snippets of code from script-to-script.  Here are a few handy ones I like to keep around.

Get the directory of the executing script

Having the directory of the executing script can be handy to load adjacent resources or as a location to which to write logs or other data:


1
$ExecutionDir = Split-Path $MyInvocation.MyCommand.Path

Dynamically add a new column to a CSV imported into a collection

Many times you need to add one or more columns to a data file you’re working on. Here’s a way to load your data file and add those other columns in one line:


1
Import-Csv "C:\somepath\some.csv" | select *, @{Name='my_new_column'; Expression={'some value'}}

Test whether an object is an object or an array

One thing I find frustrating with PowerShell is that when you retrieve an object, say through a web request or simply filtering on a collection, you don’t necessarily know the datatype of the result set. You could either have an array of objects or a single object. The problem is, the available properties change between arrays and single object. If you try to print “count” on a single object, for example, PowerShell will throw an exception. In order not to crash my scripts, then, I’ll use code like what I have below to test the datatype of my object before continuing on:


1
2
3
4
5
6
7
if($null -ne $myObj){
    if($myObj.GetType().IsArray){
        # $myObj is a collection, so deal with it as such`
    }else{
        # $myObj is a single object
    }
}

Add attributes to a XML document

Manipulating XML documents can be a real pain. Here’s an easy way to add an attribute to a XML element:


1
2
3
$x = [xml]"<top_level_element/>"
$x.DocumentElement.SetAttribute("my_attribute", "some value")
$x.OuterXml

Upload a document to a Sharepoint document library

I suspect there are probably easier ways to do this with the Sharepoint web APIs, but here’s a technique I’ve used in the past to upload a document to a document library in Sharepoint:


1
2
3
4
$web_client = New-Object System.Net.WebClient
$web_client.Credentials = [System.Net.CredentialCache]::DefaultCredentials
$my_file = gci "C:\somepath\somefile.txt"
$web_client.UploadFile( ("http://some_sharepoint_domain/sites/some_site/Shared Documents/{0}" -f $my_file.Name), "PUT", $my_file )

Use the HTML Agility Pack to parse an HTML document

Parsing HTML is the worst! In the Microsoft world, some genius came up with the HTML Agility Pack allowing you to effectively convert your HTML page into XML and then use XPath query techniques to easily find the data you’re interested in:


1
2
3
4
5
6
7
Add-Type -Path "C:\nuget_packages\HtmlAgilityPack.1.8.4\lib\Net40\HtmlAgilityPack.dll"

$hap_web = New-Object HtmlAgilityPack.HtmlWeb
$html_doc = $hap_web.Load("https://finance.yahoo.com/")
$xpath_qry = "//a[contains(@href, 'DJI')]"
$dow_data = $html_doc.DocumentNode.SelectNodes($xpath_qry)
$dow_stmt = ($dow_data.Attributes | ? {$_.Name -eq "aria-label"}).Value

Convert one collection to another (and guarantee column order)

First, imagine you have a collection of complex objects, say, a JSON document with lots of nesting. You want to try to pull out just the relevant data elements and flatten the collection to a simple CSV. This snippet will allow you to iterate through that collection of complex objects and append simplified records into a new collection. Another problem I’ve found is that techniques like Export-Csv don’t always guarantee that the columns in the resulting CSV will be in the same order you added them in your PowerShell script. If order is important, the pscustomobject is the way to go:


1
2
$col2 = @()
$col1 | %{ $col2 += [pscustomobject]@{"column1"=$_.val1; "column2"=$_.val2} }

Load multiple CSV files into one collection

It’s not uncommon to have multiple data files that you need to load into one collection to work on. Here’s a technique I use for that situation:


1
2
3
4
5
$col = @()
dir "C:\somepath" -Filter "somefile*.csv" | % { $col += Import-Csv $_.FullName }

# If you need to filter out certain files, try this:
dir "C:\somepath" -Filter "somefile*.csv" | ? { $_.Name -notmatch "excludeme" } | % { $col += Import-Csv $_.FullName }

Parse a weird date/time format

It’s inevitable that you’ll run into a non-standard date/time format that you’ll have to parse. Here’s a way to handle that:


1
$date = [datetime]::ParseExact( "10/Jun/18", "dd/MMM/yy",$null )

 

Wrapping up the school year

In the US, this is the time when schools let out for the summer–well, actually, that time began a week or two ago, but just humor me here.  Clearly, this is an important transition time from school year to summer break.  The rigor of the school schedule is gone, the “free” yellow bus transportation services are no more, late night homework and study sessions take a hiatus.  We’re now looking at the blank canvas of Summer.  Of course, as parents, our schedules don’t change and, in many ways, become harder as we struggle to find summer activities for the kids and ways to transport them there.

Nevertheless, as our children wrap up their school years, here are some actions I try to take to make the transition into Summer a little easier and more organized:

Scan, Scan, Scan

Hopefully, you’re scanning the hard copy documents, awards, and pictures your children bring home with them all throughout the year (and otherwise retaining electronic versions of report cards, etc).  Scanning, say, an award certificate as soon as your child receives it can be helpful for at least two reasons: 1) your scanner will affix the current date to your file helping you know, years from now, when your child earned that award (particularly if the award date isn’t on the certificate) and 2) the affixed date will help you organize such files so that it’s easier to include them with other, related family media.

Regardless, your children will still bring home large amounts of documents in the days before school ends.  I try to go through all these documents and scan as many as makes sense.  School photos?  Check.  Awards and certificates?  Absolutely.  Report cards, standardized test results, and the like?  Definitely.  Writing assignments, poems, and so forth?  Yep.  Going through the effort of scanning these items now can be time consuming, but is well worth it years later when the hard copies are long gone and all you have are those electronic memories.

Take Pictures

I certainly take pictures of my children at their various school events throughout the year and especially at their various end-of-year ceremonies.  I also try to take a few pictures of them in their school uniforms and at the bus stop awaiting its arrival.  I try to include their fellow travelers, as well, as some of those students may be graduating and moving on.  One could even go so far as to print up a sign saying something like, “last day for 4th grade!” and have his child hold the sign in a picture–I wouldn’t recommend that for your older children, though.

Consider filing your child’s notes and related documents

My oldest is a fantastic note taker.  On more than one occasion, we’ve been able to pull her notes out of storage and use them to help the younger ones get through a class or two the older one had already traversed.  Here’s another true story: for my first semester of college, I studied abroad in Singapore.  When I returned to campus in January, I learned that students could take placement tests to test out of certain required classes.  So, I went home and collected the two or three three-ring binders I kept of my Calculus notes from my senior year in high school.  I studied these for several days, took the placement test, and was able to test out of three semesters of math classes.  Cheap credit hours there!  Thus, retaining and organizing your child’s class notes and associated documentation could be a worthwhile endeavor.

Say, “Thanks”

“Thank yous” are free–except for the outrageously priced thank-you cards.  At any rate, your children should thank their teachers, school staff, and even the bus driver.  A card and gift might even be appropriate.  A “thank you” now might be worth a recommendation letter later.

Postmortem?

Often at work, after a project, the team will gather to do a “postmortem” or collect “lessons learned”–simply a list of things that went well, things that didn’t, and ways future projects can learn from the one just finishing.  You could consider helping your child do the same.  What did he like about the year?  What did he hate?  What teachers and classes did he enjoy and why?  Which ones did he not enjoy?  What tools did he use to study for tests and organize his assignments?  Did these work out for him or should he spend some time finding replacements?  These moments of introspection might help your child become more efficient next year.  They could help him decide on career and major as college draws near.  They could help you better coach any younger children you have following in this child’s footsteps.  You might even share your notes with parents of other, younger children entering the grade your child just completed.

Plan out the summer

What, you thought you had a break for a couple of months?  To me, summer is a great time to get a jump on the competition, er, fellow students.  I don’t know if it’s good or bad, but schools seem much more academically–and, certainly, athletically–competitive than they did when I was young.  So, while the competition is lounging by the pool, consider enrolling your children in summertime academic courses that can help them get ahead.  In summers past, we’ve enrolled our kids in Mathnasium and different sessions at the local college and community center.  Ask yourself, what are some of your child’s academic goals?

  • Competing to get into a higher math class?
  • Preparing for an upcoming standardized test?
  • Simply easing some of the anticipated pressures of the next school year?

Summertime academic courses can help meet these objectives and more.

Also new these days is the dreaded Summer homework.  Establishing a plan for your child to knock these requirements out early seems a wise thing to do.  Hopefully, you also have some idea of projects to expect in the next school year.  For example, my child will have to construct a science fair project in his next year.  I hope to spend some of this Summer at least brainstorming with him what we might try to tackle and maybe even begin gathering relevant material.  Whatever your plans are, develop and implement them early before your child gets too used to a life of leisure.

Get a teacher recommendation

If your child is already a junior or senior in high school, he either already has one or more recommendation letters or is actively working on getting one for his college applications, assuming he’s going that route.  But what if your child simply needs a recommendation for a job or maybe an internship?  Before the school doors close and the teachers disappear into the hinterlands, get a recommendation letter if you think you might need one.

Friend contact information

When you see your friends everyday at school, you may not think about how to contact them outside of the establishment.  But if your child wishes to hang out with his school chums over the break, he’ll want to make sure to get phone numbers and email addresses of them before the building shutters.

Read a good book

During the summer, kids usually can’t make the claim that they have no time to read.  Challenge your child to read a good book or two: how about one from this list?

Talk to other parents

Before the year ends, I try to chat up a parent or two of students one grade higher than mine.  What sorts of challenges should my kid be aware of next year?  What teachers and classes should we be wary of?  And so on.  Gather some intelligence on the next grade and prepare for those challenges over the break so you’re ready for them.

Sell your books?

Academic books can be outrageously expensive.  If you really think you won’t need them for reference or for a younger sibling, consider selling those books to another student and try to recoup some of your expense.

 

Any other tips I missed?  If so, please leave them in the comments!

Watermarking Jupyter Notebooks, Part 2

In a previous post, I explored different ways in which I might be able to add a watermark to my Jupyter Notebooks.  I also referenced an interesting approach that would allow me to a) watermark all my notebooks at once and b) only have to write CSS once and write it entirely outside of any notebook.  Well, I decided to go ahead and give that approach a try.  Here are my results.

Step 1: Create your custom.css

The post I found on theming your notebooks suggested that you should place your custom CSS in a custom.css file here: ~/.jupyter/custom/custom.css

The tilde is supposed to represent your home directory; although, in different installations of Jupyter Notebooks that I’ve seen, I’ve found the “home” directory for Jupyter to not necessarily be your Windows home directory.  At any rate, on my home computer, I found my Jupyter folder here: %USERPROFILE%\.jupyter

Interestingly, when I navigated to the that folder, I found no “custom” directory.  So, I created one and then created a blank custom.css within it.

Step 2: Add the necessary CSS to custom.css

Next, I opened my blank custom.css in Notepad++ and added the following CSS code (apologies for the horizontal scrolling, but I’m not entirely sure how to properly format HTML code in a CSS file):


1
2
3
4
div#notebook {
    background-image: url("data:image/svg+xml;utf8,<svg xmlns='http://www.w3.org/2000/svg' version='1.1' height='600px' width='600px'><text x='-350' y='500' fill='lightgray' font-size='80' font-family='Arial' transform='translate(30) rotate(-45 0 0)'>DadOverflow.com</text></svg>");
    background-repeat: repeat;
}

Step 3: Start up Jupyter, create a new notebook, and profit!

Now, with your newly minted custom.css, launch Jupyter Notebook and start a new notebook…or even open up an existing one.  You’ll now see your watermark magically appear:

Pretty cool, eh?

Newer posts »

© 2024 DadOverflow.com

Theme by Anders NorenUp ↑