Microsoft Azure

Azure Infrastructure Update – March 2020

Just recorded the latest Azure Infrastructure update and below are the key updates. Note a weekly update is available via subscribe action on the right hand side of this side (hint click Subscribe) or at https://savilltech.com/azure-weekly-update-subscribe/.

Lets get to it.

Azure AD B2B – Unmanaged/viral tenants will not be created after 3/31/2021. Make sure you turn on one-time-passcode (OTP).

Azure Security Center

  • Now integrated with Windows Admin Center
    • Onboard OS instances to ASC via the WAC extension
    • Security alert and recommendations surfaced
  • Identity and Access recommendations in free tier
  • Azure Container Registry scanning
  • Azure Kubernetes Service (AKS) protection

Azure Cloud Shell – Now has additional regions (secondary) for the storage of the shell persistent data. The compute will still be in one of the primary regions but now the data-at-rest can be in a region that may help you meet certain compliance requirements.

Azure Networking

  • NAT Gateway GA (https://youtu.be/c685a1CiaIs)
  • Azure Storage and Azure SQL Database Private Link GA
  • Azure Data Explorer cluster deployment into custom virtual network now possible providing integration with NSG and other connectivity to the vnet.

Azure Front Door

  • Wildcard hosts/domains
  • Configurable idle timeout
  • Configurable minimum TLS versions
  • Health probe configuration
  • Lockdown with new X-Azure-FDID
  • Disable backend certificate name check

Azure Storage – Blob immutability has GA’d. Enables WORM to blobs (write once, read many) based on legal or time locks.

Azure Dedicated Host – Now has additional hardware types available focused on general use, memory optimized and storage-intensive. All based on the AMD EPYC processor except the Msv2 new addition.

Well that’s it! Please subscribe to the YouTube channel and take care in these crazy times and see you soon!

 

Mid-March 2020 Azure Infrastructure Update

Just recorded the latest Azure Infrastructure update and below are the key updates. Note a weekly update is available via subscribe action on the right hand side of this side (hint click Subscribe) or at https://savilltech.com/azure-weekly-update-subscribe/.

Lets get to it.

PowerShell 7 was released.

New Azure AD App experience is available. It can be enabled for all users or specific groups of users.

Azure Security Center has two updates.

  • Continuous export available
    • Trigger alerts via Azure Monitor through Log Analytics export
    • Can also send to 3rd party SIEM via event hub
  • Just-in-time experience updates
    • Justification field available during access request
    • Cleanup of redundant rules that used to be left behind as part of NSGs

Azure Backup now has Backup Reports to provide tracking and auditing of backup and restore jobs.

Two new Virtual Machine skus.

  • NDv2 GPU VMs GA
    • High-end deep learning training and HPC
    • 8 NVIDIA Tesla V100 NVLink interconnected GPUs
  • NVv4 VMs GA
    • GPU accelerated graphics applications and virtual desktops
    • AMD Radeon Instinct MI25 GPU
    • Various sizes with partial GPU support

VM Scale Sets (VMSS) has a number of new capabilities.

  • Automatic repair based on application health via load balancer health probe of the application health VMSS extension
  • Scale-in policy to control how scale-in is performed for example default (based on balancing first), newestVM or oldestVM
  • Instance protection to protect specific instances during scale-in or other types of action
  • Instance termination notifications through scheduled events along with configurable delay. https://docs.microsoft.com/en-us/azure/virtual-machines/windows/scheduled-events

And finally Cosmos DB introduces a free tier that is available once per subscription giving 400 RU/s and 5GB of storage. Additionally dynamic scale is not available via autopilot which will make whatever RUs are needed in real-time available up to a defined limit. Autopilot can work with the free tier meaning you would only pay for RU/s over 400.

Well that’s it! Please subscribe to the YouTube channel and take care in these crazy times and see you soon!

 

Full Azure Data Engineer Associate Learning Track Available

Over Q2 and Q3 of 2019 I have been working on a series of courses to cover the content required to pass exams DP-200 and DP-201 that once passed award the Azure Data Engineer Associate certification. I completed the final two courses and now the 11 part learning track is available. It will shortly be available as a learning path on both Pluralsight and Azure.com but wanted to link to all 10 courses and the order that I recommend you watch them in now. They are all free. You just need to sign up for a free Pluralsight account via https://www.pluralsight.com/partners/microsoft/azure.

Note, I felt I should put my money where my mouth is so took both the exams yesterday back-to-back and pleased to say I passed them both (otherwise this would be a much sadder blog post) ;-). Also during the 45 minutes it took me to complete DP-200 there was a fire alarm test of my facility by the fire marshal and so the entire time I had a siren blaring and strobe light while trying to focus. That was fun. I felt like robin hood where he is trying to make the shot and everyone is distracting him 🙂 Also they shutdown the AC during the tests so it was 85 degrees.

azuredataengineerassociateexamsandcert

Hell’s Cloud Ops

Been watching Hell’s Kitchen in the background while working on some projects and I think it would make an awesome cloud operations show and a fun way to communicate some core concepts. Imagine…..

Chef in calm voice – OK team, today we are working on providing a tasty SQL service for our customer that will be used from a fairly basic application. Off you go.

<contestants scurry off to their workstation areas>

<chef wanders over to Bob>

Chef angry – Bob, WHAT ARE YOU DOING?

Bob – I’m creating each VM to be part of the SQL cluster I’m creating

Chef furious – You’re creating each VM one at a time in the portal???? Oh my god! Is your computer made of red and yellow plastic with “My first” written on the top of it? At least I see you’re using Availability Sets for some resiliency but this is ridiculous. How will you ensure consistency? How will you scale to creating 50 instances of this? How would this integrate with DevOps. Start again, use Infrastructure as Code and if I see you in a portal that mouse will be going where the sun doesn’t shine.

Bob – Yes chef!

<15 minutes later Bob presents his template>

Chef – OK, nice template, good resources. oh no no no no. What have you done????? WHY HAVE YOU HARD CODED values in the resources section??? WHERE IS THE PARAMETER FILE?? How are you going to change control this? How will you deploy this between different environments, deploy between different instances. You donkey! Take environment specific values out of the template and get them in a parameter file! Then you have one, change controlled template. Environment, instance specific values are completely separate! IDIOT! FIX THIS!

<5 minutes later Bob returns>

Chef – Lets see. Good parameter use, lets look at the parameter file. DONKEY! Are you here to destroy the company??? WHYYYYY do you have the administrator password in the parameter file???

Bob – I needed it to join the machines to the domain via the domain join extension chef

Chef – And you felt the best way to do that was to place that password in the file that you then uploaded to a repository??? Your companies most important password is now known to everyone and a group of teenagers has taken over your company, your wife has left you and your kids pretend they are adopted they are so embarrassed. Good luck stocking vending machines after destroying your company. IDIOT! Where would be a better place do you think? CAN YOU THINK?

Bob – Azure Key Vault chef

Chef – Can you do that? are you capable. DO IT! And heaven help you if you forget to update the vault’s advanced access policy to allow use of the secret from ARM template deployments.

<5 minutes and Bob returns>

Chef – Lets see how you can ruin my day now. This is acceptable. Will work well. Nice use of secrets. I see you even created a release pipeline. Now tell me, why didn’t you just use Azure SQL database?

<A small tear rolls down Bob’s cheek and credits roll>

Deploying Agents to Azure IaaS VMs using the Custom Script Extension

In an ideal world organizations should try to avoid creating custom images with their own special agents and configurations. This means a lot of image management as each time an agent is updated the image has to be updated in addition to the normal patching of OS instances. The Azure marketplace has a large number of OS images that are kept up-to-date which should be used if possible and any customization performed on top of that. I recently had a Proof of Concept where a number of agents needed to be deployed post VM deployment along with other configurations. Items such as domain join can be done with the domain join extension but for the other agent installs we decided to use the Custom Script Extension to call a bootstrap script which would do nothing other than pull down all content from a certain container using azcopy.exe and then launch a master script. The master script would be part of the downloaded content and would then perform all the silent installations and customization’s required.

A storage account is utilized with two containers:

  • Artifacts – This contains the master script and all the agent installers etc. This could use a zip file to enable a structure to be maintained of the various agents and the master script could unzip at the start
  • Bootstrap – This contains azcopy.exe (in my case version 10) and the bootstrap.ps1 file that does nothing other than call azcopy to copy everything from the artifacts container to the c:\ root, then launch the master script from the local copy

Below is my example bootstrap.ps1 file. Notice it has one parameter, the URI of the container which will be the shared access signature enabling access.

Azcopy.exe was downloaded from https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10 and copied to the bootstrap container along with the bootstrap.ps1 file. In my case there is nothing sensitive in the file and so I made the container public. This would avoid having to have an access key as part of my ARM template that would ultimately call this script.

All the installers and the master script were uploaded to the artifacts container. For this container I wanted a shared access signature (SAS) that would give read and list rights. The idea would be some automation would generate a new SAS each week and write to a secret in key vault that only people that should deploy had access to. The SAS would have a lifetime of 2 weeks to have an overlap with the newly generated. In addition to generating and storing the complete SAS I needed a second version that was escaped for cmd.exe. This is because the SAS has & in it which was being interpreted during my testing breaking its use. I tried to use no parse (–%) but this did not work since it was being called by cmd.exe therefore the escape is to use ^&. The script below generates the SAS and the escaped SAS and writes both versions as secrets to key vault.

Once this was done I now had a SAS available in the key vault that would give read and list to the artifacts container. Remember to configure the Access Policy on the vault to enable use of secrets from ARM templates (advanced settings) and additionally for the users/groups to have access to the secret. A test of this process to my local machine worked, i.e.

Next I tried calling as I would via the Custom Script Extension which with the escaped version worked great (note its the escaped URL as this will get expanded in the template).

Initially my test was to an existing Azure VM so I used the following (note I’m getting the escaped version of the secret from Key Vault):

Once this worked I finally created an ARM template that included a reference to the secret and all worked as planned.

The parameter file (note I also get a secret to join the domain even though I’m not using the domain join extension in this example):

The actual template (note in the CSE extension at the end I need the single quotes around the URI or it once again tries to interpret it so you have to use two, i.e. ”, to get one ‘ when it actually executes):

And the execution (note the network and RG already existed in my environment).

Hope this helps!

New Free Data Courses on Pluralsight Available!

Over the past two months I’ve been busy on some Data in Azure courses for Microsoft and Pluralsight. These are free and you just need to sign-up for a free account on Pluralsight. These will shortly be available via Azure as well but are available now through Pluralsight.

Azure Stack Marketplace Management

Been doing some work with Azure Stack and wanted to easily update all the Microsoft provided extensions and a set of core images if there are new versions by running a simple script.

Script available at https://github.com/johnthebrit/AzureStack/blob/master/azurestackmarketplace.ps1. Simply run the script and after it downloads the assets it will check if there are older versions and prompt you if you want to delete the old ones.

Lots of new Azure Design and Identity free training available

I may have seemed to be very quiet over the past few months but that’s because I’ve been working pretty much every night and weekend on 11 new courses for azure.com that will shortly be available via the site but are immediately available for free via PluralSight. If you don’t have an account simply sign up for a free account and you can then access my (and other peoples tracks).

Planning Microsoft Azure Identity and Security

Planning Microsoft Azure Infrastructure

The identity track looks at identity management before diving into authentication, authorization, auditing, monitoring and risk. The infrastructure track looks at compute, storage, networking and monitoring.

I hope you find these courses useful and there are more to come.

On a side note I’m trying to raise money for Cure Childhood Cancer as part of my Ironman Chattanooga on 9/30/2018. This will be my 5th Ironman this year and 12th overall. If you can help even a little please head over to https://www.firstgiving.com/fundraiser/john-savill/IM2018 and maybe your company matches so if they do that helps as well. I’ll be trackable on the day via https://bat.live/track/imchattanooga2018?bib=356.

Thank you!