Lots of new Azure Design and Identity free training available

I may have seemed to be very quiet over the past few months but that’s because I’ve been working pretty much every night and weekend on 11 new courses for azure.com that will shortly be available via the site but are immediately available for free via PluralSight. If you don’t have an account simply sign up for a free account and you can then access my (and other peoples tracks).

Planning Microsoft Azure Identity and Security

Planning Microsoft Azure Infrastructure

The identity track looks at identity management before diving into authentication, authorization, auditing, monitoring and risk. The infrastructure track looks at compute, storage, networking and monitoring.

I hope you find these courses useful and there are more to come.

On a side note I’m trying to raise money for Cure Childhood Cancer as part of my Ironman Chattanooga on 9/30/2018. This will be my 5th Ironman this year and 12th overall. If you can help even a little please head over to https://www.firstgiving.com/fundraiser/john-savill/IM2018 and maybe your company matches so if they do that helps as well. I’ll be trackable on the day via https://bat.live/track/imchattanooga2018?bib=356.

Thank you!

Easily create multiple subnets in an Azure Virtual Network

I recently needed to create a whole set of subnets in a large number of virtual networks of various sizes. I thought some variables would be a great way to quickly create the set of subnets in each virtual network which were each /20 networks in a shared class B IP which enabled 16 virtual networks per Class B IP space. The goal was to show that each subnet didn’t need to be a full class C (/24) in instead we could use smaller subnets based on the number of hosts actually required. I’ve included the comments which explains the subnets created and the number of hosts supported in each.



Use an Application Image from the Azure Marketplace using PowerShell

I recently needed to deploy a special type of VM from the Azure Marketplace using PowerShell and the deployment was not the same as regular Windows or Linux VM.

First I knew the app I wanted to use, e.g. https://azuremarketplace.microsoft.com/en-us/marketplace/apps/microsoft-ads.windows-data-science-vm but wasn’t sure of the publisher or offer. With hindsight it shows you right in the URL, Microsoft-ads is the publisher and windows-data-science-vm is the offer but I initially just searched for what I wanted using the following and looked for it (first part of the code), then got the detail as usual (last two lines):

Now I knew the publisher, offer and SKU I could add that configuration to my VM.

However, when using the application there are a few other steps. You need to set a plan and also accept the terms of the app. Fortunately it’s easy to do.

That was it. Basically the only additional work is setting the plan and accepting the marketplace terms and the data needed for those commands are the same values used for the source image, just PublisherName > Publisher, Offer > Product and SKUs > Name. The exact same would apply if using JSON. Easy!

Deploying an Azure IaaS VM using PowerShell

I recently had to deploy some new VMs and wanted to use PowerShell and also join them to a domain and get the anti-malware extension used. Below is the PowerShell I used. You would need to modify the variables in the below to match your own domains.


Checking the creation time of an Azure IaaS VM

I recently had a requirement to check the age of VMs deployed in Azure. As I looked it became clear there was no metadata for a VM that shows its creation time. When you think about this it may be logical since if you deprovision a VM (and therefore stop paying for it) then provision it again what is its creation date? When it was first created or when it last was provisioned.

As I dug in there is a log written at VM creation however by default these are only stored for 90 days (unless sent to Log Analytics) BUT if its within 90 days I could find the creation date of any VM. For example:

What if the VM was not created in the last 90 days? If the VM uses a managed disk you can check the creation date of the managed disk. If it is NOT using a managed disk then there is not creation time on a page blob however by default VHDs include the creation data and time as part of the file name. Therefore to try and find the creation data of a VM I use a combination of all 3 by first looking for logs for the VM, then look for a managed disk and finally try a data in unmanaged OS disk.


Deploying Operating Systems in Azure using Windows PE

In this article I want to walk-through deploying operating systems in Azure using a custom Windows PE environment and along the way cover some basics around PE and OS deployment. Before going any further I would stress I don’t recommend this. The best way to deploy in Azure is using templates, have generic images and then inject configuration into them using declarative technologies such as PowerShell DSC, Chef or Puppet however there are organizations that have multiple years of custom image development at their core that at least in the short term need to be maintained which was my goal for this investigation. Is it even possible to use your own Windows PE based deployment.

My starting point was to get a deployment working on-premises on Hyper-V. Azure uses Hyper-V and at this level there really is nothing special about what Azure does so my thinking is if I got a process running on-premises I should be able to take that VHD, upload it to Azure, make an image out of it and create VMs from it (and this proved to be true!). The benefit of this approach was speed of testing and the ability to interact with the Windows PE environment during the development and testing phase. Something that is much harder in Azure as there is no console access.

The first step was to create a VHD (not VHDX for Azure compatibility) that contained Windows PE which I would boot to. I downloaded the latest Windows ADK from https://developer.microsoft.com/en-us/windows/hardware/windows-assessment-deployment-kit (1709) and installed on a machine. Once installed I created my own Windows PE (x64) instance via the Deployment and Imaging Tools Environment. I used the following commands:

I then wanted to add PowerShell and other components to it including imagex.exe:

Notice in the code above when I’m adding packages I do this my mounting the boot.wim file that is part of my copied PE environment, performing actions against it then committing those changes when I unmounts it. I’m modifying that boot.wim. This is an important point.

Once the PE was ready I wanted to quickly test so I built a VHD based on that PE environment.

This creates a new VHD file and attaches it to the current OS as drive V:. I then make bootable media of my PE folder to the V: folder then detach. I then copied this VHD file to a Hyper-V box and created a VM that used it as its boot disk. Sure enough it booted and I was facing a PE environment. The next step was to format the disks and apply an image automatically. My initial though was “how can I format the disk and apply an OS to the disk if booted from it (PE)?” however quickly it became obvious that the PE I was booted into wasn’t really running from the local disk. Instead what happens is on boot the boot.wim file on the PE media is read into a writable RAM disk which is where the PE actually runs from (the X drive). Therefore even though the C: drive contained that boot.wim it’s not actually being used and do it can be wiped. Therefore I created a script that did three things

  1. Wiped the disk and create the system and windows partitions
  2. Applied a Windows Server image (1709 Server Core)
  3. Make the disk bootable

To partition the disk I created a text file, parts.txt which contained:

I could then call this with (I would copy this to my Windows PE environment as well):

The WIM file I placed on a file share (this would be an Azure Files share once in Azure) so I had to map to the network drive and apply so the complete file became:

I saved this as autolaunch.bat and added to the root of my Windows PE boot.wim (by remounting it) along with the parts.txt. I also modified the startnet.cmd found under the WindowsSystem32 folder of my mounted PE environment to call my autolaunch.bat file, e.g.

I then unmounted and created a new VHD. I made sure my install.wim was present in my file server as referenced, copied over the VHD to my Hyper-V server, changed the VM to use the new VHD and sure enough it booted, formatted the disks and laid down the image. Note you are putting a password in the file, this is not ideal. Also not if your password contains special characters you may have to escape them in the batch file or they won’t work correctly, for example if you password contained % you actually need %% in the string!

The next step was to try this in Azure. I created a storage account in Azure, added an Azure Files share and uploaded the install.wim file to it. I changed the autolaunch.bat to map to the Azure Files share instead of the local file share (along with the path to the WIM file). It therefore became:

And execute to:

To upload the VHD to Azure and create an image from that uploaded file I used the following PowerShell. This is important. Trying to upload from other tools or the Azure portal seems to leave the VHD in a strange state and unusable.

From here I created a new VM from my image. I used PowerShell below to create my VM. Note I’m enabling boot diagnostics. This allowed me to view the console even if I couldn’t interact with it. Therefore I had some idea what was happening

I then jumped over to the portal and via the Support + Troubleshooting section – Boot diagnostics – Screenshot I could see it deploying in Azure (updated about every 30 seconds or so).


This worked! The OS installed and strangely I could RDP to it even though I never enabled this, it had the right name, it had the Azure agent installed. What trickery is this and then it hit me. I never added my own unattend.xml file. All I did was apply a 2016 image to a disk and it rebooted. Basically the same as if I had used a template with 2016. The ISO file that Azure automatically creates when deploying a VM that contains an unattend.xml file and other setup files still got created, still got attached and was therefore still used. This was good but also bad as I wanted to use my own unattend.xml file to further prove we could customize.

The next step was to generate my own unattend.xml file and use it. At this point I didn’t want to keep having to rebuild the VHD every time I made a script change and so I broke apart the logic so the autolaunch.bat just connected to the Azure Files, partitioned the disk then copied down an imageinstall.bat file and executed. This way I could change imageinstall.bat on the file share whenever I wanted to change the functionality. autolaunch.bat became:

And imageinstall.bat which was placed on the file share became:

I created a new VHD with the reduced autolaunch.bat and uploaded to Azure (and created a new image after deleting the old one with Remove-AzureRmImage -ImageName $imageName -ResourceGroupName $rgImgName).

Now I’m jumping over a few steps here but basically I created an unattend file to set a default password, have a placeholder for the computer name, enable auto mount of disks, move the pagefile to D:, enable RDP and required firewall rules and also launch the install.cmd that Azure normally runs. This would install the agent, register with Azure fabric etc. Because I place my unattend.xml in the windows\panther folder it overrides any found on removable media, i.e. the Azure one! My unattend file was:

Now in this file I have a placeholder string for the computername, TSCAEDPH . I wanted to replace this with the computername specified on the Azure fabric. How would I get this from inside the guest? Well Azure has an endpoint at that can be called from within a VM and basic information can be found and so I created a PowerShell script that would find the computername and update the unattend.xml I had copied to the panther folder of the deployed image:

This was saved as unattendupdate.ps1 on the Azure Files share as well which now contained the install.wim, unattend.xml, imageinstall.bat and this ps1 file. Fingers crossed I kicked off a new VM build. It worked. It used my unattend.xml file but still got the Azure agent etc installed. It also still renamed the local administrator account to that specified as part of the VM creation as that happens as part of the Azure install step process which I was now calling from my unattend.xml file.

Now there are some problems here. If Azure changes the structure of their ISO file with the install.cmd this will break so it would have to be re-investigated however this is still better than trying to duplicate everything they do manually which is far more likely to change far more often.

So there you go. You can use your own PE in Azure to customize and create deployments including unattend. You can still call the Azure agent install and finalize. But ideally, use images 😉

Automating deployments to Azure IaaS with custom actions

Firstly the final scripts of all the content discussed are available here. A video walkthrough is available at https://youtu.be/7bobbg91cQc and included below.

In this post I want to document the results of a POC (proof of concept) I was engaged in for a very large customer. The customer wanted to create single/multi VM environments in Azure for dev/test/QA purposes. The goal was a single command execution that would create the VM and in this case make it a domain controller, install SQL Server 2012 then install SharePoint 2010. For this scenario I decided to use PowerShell rather than JSON just to demonstrate the PowerShell approach since there are already many JSON templates in the gallery around SharePoint deployment.

To enable this solution the high level workflow would be:

  1. Create a new resource group and in that create a new storage account and virtual network (since each environment was to be isolated and by placing in their own resource group the lifecycle management, i.e. deletion, would be simple)
  2. Create a new VM using the created resources
  3. Execute PowerShell inside the VM via the Azure VM Agent to promote the VM to a domain controller then reboot it
  4. After the reboot execute additional PowerShell to create accounts, open firewall exceptions, install SQL Server 2012 then install SharePoint 2010

The unattended installation of a domain controller via PowerShell is very simple. Below is an example that creates a pocdom.local domain.

You will notice in the code I write the AD to the E: drive. This is because in Azure the OS disk by default is read/write cache enabled which is not desirable for databases. Therefore for the VM I add two data disks with no caching; one for AD and one for SQL and SharePoint. The code below is what I use to change the drive letter of the DVD device then initialize and format the two data disks.

The two pieces of code above would be combined into the first boot PowerShell code (with the disk initialization block before the DC promotion code). Once the reboot has completed firewall exceptions for SQL and SharePoint need to be enabled.

Next I need the SQL Server and SharePoint media along with unattended commands to install. I decided to use Azure Files as the store for the installation media. Azure Files presents an SMB file share to the VMs with only the storage account key and name required to access. In my example I place this in the PowerShell script however it could also be injected in at runtime or stored more securely if required. Create a storage account then create an Azure Files share through the portal and take a note of the access key and storage account name.

Into this share I will copy the SQL Server and SharePoint installation files. The easiest way to upload content is using the free Azure Storage Explorer tool from http://storageexplorer.com/.

Now the details of performing unattended installations of SQL and SharePoint are outside the scope of this write-up as the goal for this is more how to install applications through Azure IaaS PowerShell however at a very high level:

  • To install SQL Server unattended simply requires a configuration file which can be generated by running through the graphical SQL Server setup and on the last page it will show you the location of the configuration file it will use for installation. Simply copy this file and cancel the installation. Copy the SQL Setup structure and the configuration file to the Azure Files share. I place the ConfigurationFile.ini in a separate Assets folder on the share. Then use that setup file with the SQL setup.exe, for example
  • For the SharePoint unattended installation I used the autospinstaller solution which is fully documented at https://autospinstaller.com/ and includes a web based site to create the unattended answer file used by the program. Follow the instructions on the site and copy the resulting structure to the Azure Files share.

My resulting Azure Files share consists therefore of 3 folders:

  • AutoSPInstaller – The SharePoint installation media and AutoSPInstaller solution
  • POCAzureScripts – The SQL configuration script
  • SQLServer2012SP3 – SQL Server installation media

To map to the share, copy the content, trigger the SQL Server installation from the share, dismount the share then trigger the SharePoint installation I use the following (which also adds an account named Administrator as that was a requirement). I would add the firewall exception creation to this code as the secondboot PowerShell file. You will notice I wait for 40 minutes at the end for the SharePoint installation to complete. I run the SharePoint install as a separate, asynchronous job as at the end it asks for key presses to continue so this avoids trying to handle that and after a reboot that will all get cleared up.

At this point I have a firstboot.ps1 and a secondboot.ps1 file. Upload those files into blobs in a container named scripts in the same storage account as the Azure Files. These files will be used as part of the total VM provisioning process.

The final part is to create the VM and use the PowerShell created. In the example code below I create all the resources and use premium storage accounts to maximum performance however any of these parameters can be changed to meet requirements. In the code replace the <storage account name for assets> with the storage account created holding the Azure Files and blob content along with its key. Also change the VM name to something unique since a public IP name will be generated based on this name. If you will deploy this many times add some logic to include some random sequence or perhaps the requesting username. Also include that as part of the resource group, storage account etc name.

In this example I give the VM a public IP so it can be accessed externally and has no NSG to lock down traffic. In reality you may not want the public IP and may add the environment to existing networks with connectivity to on-premises so would connect via private IP but I added public IP to handle worst case connectivity. If you do add a public IP like this example don’t use administrator account and don’t set simple passwords and make sure you configure NSGs to at least lock down traffic. I talk about NSGs at http://windowsitpro.com/azure/network-security-groups-defined and below is example ARM PowerShell to create and add an NSG to a NIC.

Finally if you want to delete the entire environment just run:


Creating a Minecraft server using the new Azure portal

A couple of years ago I wrote a basic set of instructions on creating a Minecraft server in Azure. I felt it was time to create a new set of instructions based on the new Azure portal. You need an Azure subscription to follow this tutorial, this could be a subscription you pay for, Azure benefits that are part of an MSDN subscription or even an Azure trial subscription which can be signed up for from http://azure.microsoft.com/en-us/pricing/free-trial/.

Once you have a subscription follow the steps outlined below:

Login to the new Azure portal at https://portal.azure.com. When you first login you will be at the Azure Startboard. Also notice the hub menu on the left hand side which enables access to various resources within Azure. At the bottom of the hub menu is a New button which enables all the different types of Azure service to be created.


Click the New button and under Browse select Compute which opens up the Compute blade with a list of images. Select the Windows Server 2012 R2 Datacenter image.


The Create VM blade will open which enables the configuration for the new VM to be selected. Notice in the Create VM blade there are some basic properties to be completed such as the name for the VM, a username and a password. Additionally there are options for the size of the VM, optional configurations and location.  The hostname can be anything you wish, for example Minecraft. Enter a username, it cannot be Administrator as this is reserved but could use localadmin. Enter a complex password which needs to be at least 8 characters with a mix of three of the following; lowercase, uppercase, numbers and symbols.

Select the Pricing Tier part to open the pricing tier lens which enables the size of the VM to be selected. The A1 Basic is sufficient for basic testing however if you wanted to host more players the Standard A2 may be a better fit. Standard tier VMs have higher storage IOPS and network performance in addition to load balancing and scale capabilities (which we don’t use for our single instance Minecraft server). Notice there is a View all link to show all the available VM sizes. Select a VM size and click Select.


By default a new virtual network is created for the VM along with a new cloud service which owns a Virtual IP that is accessible from the Internet and a random name is selected for the cloud service DNS name. Select Optional Configuration – Network – Domain Name – Create new domain name and type in a name. The name must be unique across all of Azure. You could try Minecraft-<your name> and click OK. As you type the name it will be checked to ensure its unique. This name will be how you can connect to your server, <name>.cloudapp.net. Click OK to the Network blade. Notice by default a new Storage Account will be created to store the virtual hard disks for the new VM however you could change this if desired.


On the main Create VM blade select the Location part. This enables the Azure region to be selected where the new VM will be created. Notice there are Azure regions throughout the world so pick one closest to you (or your players!). Close the blade. By default a checkbox is selected “Add to Startboard” that would add the new VM to the Startboard of your subscription. Click Create to create the VM.

The VM will now be created and will take around 5 minutes to be fully provisioned and ready for use. By default the VM will have an OS disk that is 127 GB in size and a temporary disk, the D: drive. Never put any data you care about on the D: drive as this is not persistent and by default will only be used for the pagefile. The OS disk has read and write caching enabled. You can also add data disks which have configurable caching options including no caching which is what is needed for databases and other types of workload that need writes to be persisted directly to disk. We will add a data disk to our Minecraft server for our Minecraft binaries and data files. You will need to wait for the VM to be created before adding the data disk. Select the VM which will open the VMs build and under the Essential lens (a lens is a group of parts that share a common theme) select the All settings link which opens the Settings blade.

Select Disks in the Settings blade and select the Attach New action. By default the maximum size of 1023 GB for a disk is selected with caching disabled. Select the Storage Container part – Choose Storage Account and select the storage account that you used for the Minecraft server. For the Container select the default vhds container. Click OK to create the new data disk. You may wonder why 1023 GB since you have to pay for storage in Azure and your Minecraft world may only be 50 MB which means it would seem you are paying for a lot of wasted space. This is not the case as Azure actually uses sparse storage which means even though you are creating a 1023 GB VHD file in Azure Storage behind the scenes storage is only actually allocated for the data written which is what you pay for.


Your VM is now created and has a data disk added to it. The next step is to connect to the new VM. In the VMs blade select the Connect action. This will download an RDP file which can either be opened or saved to disk so the exact options such as display size etc can be changed. The RDP file is populated with the DNS name of the cloud service that contains the Minecraft VM and the port for the RDP endpoint for the specific VM.


You are now connected to your Azure VM. Open Explorer and you will see your OS C: drive and the temporary storage drive D: but the data disk we added is not shown because it has not yet been initialized or formatted. Open the Disk Management MMC snap-in (Start – Run – diskmgmt.msc). When the snap-in opens it will inform you of a new disk and offer to initialize. Click OK. Once the disk is online right click on the disk and select New Simple Volume. Except all the defaults. Notice on the Format Partition dialog enter a label for the Volume label such as Data and make sure “Perform a quick format” is selected. This is critical in Azure as remember that sparse storage? If you don’t perform a quick format every block of the disk will be written to which means you would then pay for the full 1023 GB size. Complete the dialogs to create the new data disk.


Navigate back to Explorer and the data disk will now be visible. Select the data disk and create a folder called Minecraft.

Open Internet Explorer and navigate to https://minecraft.net/download. In the Multiplayer Server area download the latest server binary and save to the Minecraft folder that you created on the data disk. I normally rename this download to minecraft_server.exe and remove the version number from the name. You also need to download and install Java. This can be downloaded from https://www.java.com/en/download/manual.jsp and select the 64-bit version. During the Java installation you likely want to unselect the options to install and set Ask as the default!

You are now ready to get Minecraft running. I recommend creating a batch file to launch the Minecraft server which will configure it to use more memory. I save the following to a file (minecraft.bat) and place in my Minecraft folder. This launches Minecraft server and set it to use 2 GB of memory:

“C:Program FilesJavajre1.8.0_45binjavaw.exe” -Xms2048m -Xmx2048m -jar “Minecraft_Server.exe”

Run the batch file. Once it has run open the eula.txt and change the false to true then rerun the minecraft.bat file which will now launch the Minecraft server service however it is not usable yet.

Minecraft clients communicate to the server on port 25565 which by default is blocked by the Windows Firewall. You need to create a firewall exception. Click Start and type firewall. This will find the Windows Firewall with Advanced Security application. Launch it. Select Inbound Rules and select the New Rule action. Select a type of Port in the New Inbound Rule Wizard and click Next. In the next page select TCP and type in port 25565 then click Next. Accept the defaults to Allow and for all types of profile and on the final page enter a name of Minecraft Server. Click Finish.


There is one final action. The VM created sits within a cloud service and the cloud service has the publically accessible IP address. Endpoints are created on that IP address which enable communications on specific ports to be forwarded to specific ports on VMs in the cloud service. You already used one of these endpoints when you RDP’d to the VM earlier, that is using an automatic endpoint that was created to enable RDP access to the VM from the Internet. We will add a new endpoint for the Minecraft port. Open the VMs blade in the Azure portal and select All settings. Select Endpoints where you will see the existing endpoints created. Click Add. Enter a name of Minecraft and set the public and private port to 25565 then click OK.


You are now ready to use your new Minecraft server. Launch the Minecraft client and select Multiplayer. Click Add Server and for the Server Address use your cloud server DNS name, e.g. minecraft-savill.cloudapp.net and click Done.


Select your new server and click Join Server.


And play!


As an optional step you probably want to make yourself an operator for your server. Add to the ops.json file. http://conoroneill.net/creating-a-valid-ops-file-in-json-format-for-minecraft-179 walks through this process and links to http://minecraft-techworld.com/uuid-lookup-tool to find your UUID for your account.