Musings of a dad with too much time on his hands and not enough to do. Wait. Reverse that.

Category: technology (Page 32 of 36)

Handy PowerShell snippets

I code a fair amount with PowerShell at work and home and find myself reusing different snippets of code from script-to-script.  Here are a few handy ones I like to keep around.

Get the directory of the executing script

Having the directory of the executing script can be handy to load adjacent resources or as a location to which to write logs or other data:


1
$ExecutionDir = Split-Path $MyInvocation.MyCommand.Path

Dynamically add a new column to a CSV imported into a collection

Many times you need to add one or more columns to a data file you’re working on. Here’s a way to load your data file and add those other columns in one line:


1
Import-Csv "C:\somepath\some.csv" | select *, @{Name='my_new_column'; Expression={'some value'}}

Test whether an object is an object or an array

One thing I find frustrating with PowerShell is that when you retrieve an object, say through a web request or simply filtering on a collection, you don’t necessarily know the datatype of the result set. You could either have an array of objects or a single object. The problem is, the available properties change between arrays and single object. If you try to print “count” on a single object, for example, PowerShell will throw an exception. In order not to crash my scripts, then, I’ll use code like what I have below to test the datatype of my object before continuing on:


1
2
3
4
5
6
7
if($null -ne $myObj){
    if($myObj.GetType().IsArray){
        # $myObj is a collection, so deal with it as such`
    }else{
        # $myObj is a single object
    }
}

Add attributes to a XML document

Manipulating XML documents can be a real pain. Here’s an easy way to add an attribute to a XML element:


1
2
3
$x = [xml]"<top_level_element/>"
$x.DocumentElement.SetAttribute("my_attribute", "some value")
$x.OuterXml

Upload a document to a Sharepoint document library

I suspect there are probably easier ways to do this with the Sharepoint web APIs, but here’s a technique I’ve used in the past to upload a document to a document library in Sharepoint:


1
2
3
4
$web_client = New-Object System.Net.WebClient
$web_client.Credentials = [System.Net.CredentialCache]::DefaultCredentials
$my_file = gci "C:\somepath\somefile.txt"
$web_client.UploadFile( ("http://some_sharepoint_domain/sites/some_site/Shared Documents/{0}" -f $my_file.Name), "PUT", $my_file )

Use the HTML Agility Pack to parse an HTML document

Parsing HTML is the worst! In the Microsoft world, some genius came up with the HTML Agility Pack allowing you to effectively convert your HTML page into XML and then use XPath query techniques to easily find the data you’re interested in:


1
2
3
4
5
6
7
Add-Type -Path "C:\nuget_packages\HtmlAgilityPack.1.8.4\lib\Net40\HtmlAgilityPack.dll"

$hap_web = New-Object HtmlAgilityPack.HtmlWeb
$html_doc = $hap_web.Load("https://finance.yahoo.com/")
$xpath_qry = "//a[contains(@href, 'DJI')]"
$dow_data = $html_doc.DocumentNode.SelectNodes($xpath_qry)
$dow_stmt = ($dow_data.Attributes | ? {$_.Name -eq "aria-label"}).Value

Convert one collection to another (and guarantee column order)

First, imagine you have a collection of complex objects, say, a JSON document with lots of nesting. You want to try to pull out just the relevant data elements and flatten the collection to a simple CSV. This snippet will allow you to iterate through that collection of complex objects and append simplified records into a new collection. Another problem I’ve found is that techniques like Export-Csv don’t always guarantee that the columns in the resulting CSV will be in the same order you added them in your PowerShell script. If order is important, the pscustomobject is the way to go:


1
2
$col2 = @()
$col1 | %{ $col2 += [pscustomobject]@{"column1"=$_.val1; "column2"=$_.val2} }

Load multiple CSV files into one collection

It’s not uncommon to have multiple data files that you need to load into one collection to work on. Here’s a technique I use for that situation:


1
2
3
4
5
$col = @()
dir "C:\somepath" -Filter "somefile*.csv" | % { $col += Import-Csv $_.FullName }

# If you need to filter out certain files, try this:
dir "C:\somepath" -Filter "somefile*.csv" | ? { $_.Name -notmatch "excludeme" } | % { $col += Import-Csv $_.FullName }

Parse a weird date/time format

It’s inevitable that you’ll run into a non-standard date/time format that you’ll have to parse. Here’s a way to handle that:


1
$date = [datetime]::ParseExact( "10/Jun/18", "dd/MMM/yy",$null )

 

Watermarking Jupyter Notebooks, Part 2

In a previous post, I explored different ways in which I might be able to add a watermark to my Jupyter Notebooks.  I also referenced an interesting approach that would allow me to a) watermark all my notebooks at once and b) only have to write CSS once and write it entirely outside of any notebook.  Well, I decided to go ahead and give that approach a try.  Here are my results.

Step 1: Create your custom.css

The post I found on theming your notebooks suggested that you should place your custom CSS in a custom.css file here: ~/.jupyter/custom/custom.css

The tilde is supposed to represent your home directory; although, in different installations of Jupyter Notebooks that I’ve seen, I’ve found the “home” directory for Jupyter to not necessarily be your Windows home directory.  At any rate, on my home computer, I found my Jupyter folder here: %USERPROFILE%\.jupyter

Interestingly, when I navigated to the that folder, I found no “custom” directory.  So, I created one and then created a blank custom.css within it.

Step 2: Add the necessary CSS to custom.css

Next, I opened my blank custom.css in Notepad++ and added the following CSS code (apologies for the horizontal scrolling, but I’m not entirely sure how to properly format HTML code in a CSS file):


1
2
3
4
div#notebook {
    background-image: url("data:image/svg+xml;utf8,<svg xmlns='http://www.w3.org/2000/svg' version='1.1' height='600px' width='600px'><text x='-350' y='500' fill='lightgray' font-size='80' font-family='Arial' transform='translate(30) rotate(-45 0 0)'>DadOverflow.com</text></svg>");
    background-repeat: repeat;
}

Step 3: Start up Jupyter, create a new notebook, and profit!

Now, with your newly minted custom.css, launch Jupyter Notebook and start a new notebook…or even open up an existing one.  You’ll now see your watermark magically appear:

Pretty cool, eh?

Watermarking Jupyter Notebooks

Watermarked content can be annoying, but I get it: you work hard on a product and you should get credit for it. At my job, I sometimes have the same challenges–I work hard on a report or product, hand it over to the co-worker who requested it, and he takes it to upper management passing the content off, either unintentionally or otherwise, as his own. If only I could in some way watermark my work to make it clear where it originated.

Since I do a fair amount of work these days in Jupyter Notebook, I decided to take this question there and came up with three approaches:

1. The traditional image file approach

The most common approach to watermarking a web page seems to be creating an image file with your watermark and referencing that image file in your CSS. In Jupyter Notebook, you can use the IPython API to load your CSS:


1
2
3
4
5
6
7
8
from IPython.core.display import HTML


HTML('<style type="text/css">'
    'div#notebook {'
    'background-image: url("dadoverflow_wm.png");'
    'background-repeat: repeat;}'
    '</style>')

(The full notebook is here)

Alternatively, you can use this clever custom.css approach so that your watermark loads automatically in all your notebooks without any extra coding on your part.

One big drawback to this solution is that if you hand off your notebook to your customer, that image file needs to go with it. If your customer wants to get rid of your watermark, all he has to do is throw away the image file. Bummer!

2. Base64 encode that image file

Files can be embedded in a web page, so to speak, by base64 encoding them and pasting the encoding right into your HTML. You can take this approach with your watermark and then never have to worry about the image file having to travel side-by-side with your notebook. Here, I first base64 encode my image file and then use the IPython API to shove my markup, including my base64, into the CSS:


1
2
3
4
5
6
7
8
9
10
11
12
13
import base64
from IPython.core.display import HTML


with open('dadoverflow_wm.png', 'rb') as f:
    encoded_string = base64.b64encode(f.read()).decode()
   
image_url = 'data:image/png;base64,{0}'.format(encoded_string)
HTML('<style type="text/css">'
    'div#notebook {'
    'background-image: url("' + image_url + '");'
    'background-repeat: repeat;}'
    '</style>')

(The full notebook is here)

Same deal as above, you can also paste that markup in a custom.css that all your notebooks will inherit. While this eliminates your dependence on that image file, base64 strings, like this one, are huge! It is not fun dealing with such things. Fortunately, there’s yet a better solution.

3. The SVG way

SVG tags are the continuum transfunctioners of HTML–their mystery is only exceeded by their power. But you can do amazing things with SVGs, watermarking your pages among them. Here, I don’t even need to rely on an image file, base64 encode or not. I simply adjust the properties of my SVG and TEXT tags as needed to get the effect I’m after:


1
2
3
4
5
6
7
8
9
10
from IPython.core.display import HTML


svg = ("<svg xmlns='http://www.w3.org/2000/svg' version='1.1' height='600px' width='600px'>"
       "<text x='-350' y='500' fill='lightgray' font-size='80' font-family='Arial' transform='translate(30) rotate(-45 0 0)'>DadOverflow.com</text></svg>")
HTML('<style type="text/css">'
    'div#notebook {'
    'background-image: url("data:image/svg+xml;utf8,' + svg + '");'
    'background-repeat: repeat;}'
    '</style>')

 

You can find all three of these examples in my GitHub page.

« Older posts Newer posts »

© 2025 DadOverflow.com

Theme by Anders NorenUp ↑