Thursday, January 5, 2017

Domain join Azure VM using Azure Automation DSC

Abstract

Azure automation has changed a lot since I wrote last blog about AutoShutdown of Azure VMs using Azure Automation. Looking at the phenomenal rate of Azure platform evolution it makes perfect sense to revisit same services and write a new blog with absolutely new feature and tasks.
This article highlights step by step guide to make an Azure VM domain joined automatically using Automation DSC feature. This guide does not cover
-        Step by step flow on creating Azure Automation account in Azure Portal.
-        Azure VM provisioning
-        Domain configurations on domain controller

What is DSC?

DSC stands for Desired State Configuration. It’s a configuration management tool. There are many configuration tools available in the market. Few popular names are Chef and Puppet. DSC is also configuration management tool from Microsoft. Basically, it helps to automate tasks which would be very boring to do manually otherwise.
Example of such a boring task is, domain join the Azure VM when it is provisioned. I am working with one of the customer where almost every month they provision 100+ VMs on Azure and remove them. To satisfy the organization compliance and security policies all VMs should domain joined. Poor IT team had to do this domain joining repetitive task almost every day manually. There was a dedicated team member for this. He was about to go under psychiatric treatment. Thanks to Azure Automation DSC, he is back to normal now.
If interested more in knowing about DSC then link is here - https://msdn.microsoft.com/en-us/powershell/dsc/overview.
Note -
As of today Azure supports Classic(ASM) and ARM (Azure Resource Manager) type of deployments of resources. ARM is the future and this articles talks about ARM based resources only. Provisionof Azure ARM VM and configuring domain controller is out of scope of this article. Refer article - http://www.dotnetcurry.com/windows-azure/1145/active-directory-adfs-azure-virtual-machine-authentication-aspnet-mvc to understand quick steps about domain controller provisioning. The article talks about classic VM provisioning, which you can ignore and directly follow steps from section “Configure Active Directory” to promote the VM as domain controller.

Provision Azure Automation Account

Below link specifies the steps to provision Azure Automation account – CreateAzure Automation account. I am using below values for the same –



In above screenshot, subscription name is blurred; because your subscription name will be different from me and I want to keep it secret for security purpose. sssssshhhh…
New automation account will look as below -



To know about meaning of various options in Automation account like Runbooks, Assets, Hybrid Worker Groups and all refer - https://mva.microsoft.com/en-US/training-courses/automating-the-cloud-with-azure-automation-8323?l=C6mIpCay_4804984382.
As our focus is specifically on writing DSC script to make VMs auto domain join I will not spend time on various concepts and information related to Azure Automation.
With this let’s move forward to actual implementation.

Import xDSCDomainJoin module

xComputerManagement is the DSC module which can be used to make a computer domain joined. xDSCDomainjoin is stripped version of the same. This module is available on PowerShell Gallery. The central repository of PowerShell is known as PowerShell gallery. To know more refer - https://www.powershellgallery.com/.
So this PowerShell gallery has xDSCDomainjoin module and we must first import in our automation account before we use it in our script. The best way to import a module in Automation Account is from Azure Portal.
On the Azure Portal, select your Automation account. The click Assets -> Modules. All existing modules will be shown as below –



Click on “Browse Gallery” option. Search xDSCDomainjoin in the search box and it will be appear as shown below. The click on “Import” and then click Ok to complete importing procedure of module in the automation account. –



A message will appear as “Activities being extracted”. Let this procedure continue. After successful import the assets count will increase by 1 on the main page of Automation account.

Installing Azure PowerShell on local machine

On your local machine/ laptop open PowerShell ISE. You need all Azure PowerShell commands available on your local machine. Working on Azure without PowerShell is like Superman without Powers (or underwear…). Therefore, first install Azure PowerShell as per the guide given here - https://docs.microsoft.com/en-us/powershell/azureps-cmdlets-docs/#install-and-configure.

Writing DSC script for domain join

Now after installation first we must provide authentication information of Azure account to current open PowerShell ISE window. For this run the command –
Add-AzureRmAccount

This will prompt for login. Go ahead and login to complete the authentication.
Create new file in PowerShell ISE and save it as DomainJoinConfiguration.ps1. Write below PowerShell in the same file –
#first import below configuration in Azure automation account xDSCDomainjoin

Configuration DomainJoinConfiguration
{   
    Import-DscResource -ModuleName 'xDSCDomainjoin'
   
    #domain credentials to be given here   
    $secdomainpasswd = ConvertTo-SecureString "YourDomainPassword" -AsPlainText -Force
    $mydomaincreds = New-Object System.Management.Automation.PSCredential                       ("UserName@Domain", $secdomainpasswd)
   
        
    node $AllNodes.NodeName   
    {
        xDSCDomainjoin JoinDomain
        {
            Domain = 'YourDomain'
            Credential = $mydomaincreds
           
        }
    }
}

In above script replace YourDomainPassword, UserName@Domain, 'YourDomain' values by your own values. This is your final script to make an Azure VM domain joined. Now we must upload this file on Azure automation account. Therefore, click on DSC Configuration -> Add Configuration as shown below –



On the next window upload the file we created in above step and then click on Ok to complete the configuration of domain join DSC. And yes, please provide some meaningful description as shown below -




Why password in plain text?

In above script, you must have observed below line -
$secdomainpasswd = ConvertTo-SecureString "YourDomainPassword" -AsPlainText -Force

This forces to keep the password as plain text. As you have guessed this is not good practice. But I am not going to leave it here. Please read out next sections to understand why we are keeping the password in plain text. So, hold on your emotions.

Adding Configuration Data to DSC script of Domain Join

Configuration data allows you to separate structural configuration from any environment specific configuration while using PowerShell DSC.

This way, we want to separate “WHAT” from “WHERE”. DSC script we have written above specifies the structural configuration (what). This is where we define “What is needed” and does not change based on environment. Irrespective of environment; whether development or production, we want VMs to be Domain Joined. Environmental configuration specifies the environment in which the configuration is deployed (where). For example, we need common settings for all nodes and specific settings for specific nodes.

To specify environment configuration, we use “Config Data” and then we compile entire DSC script using config data. This should contain a key “All Nodes” where you specify all common configurations for all nodes that wishes to get domain joined automatically and then it can contain other node specific keys. By the way, Azure VMs we add to DSC configuration are termed as “Nodes”.
For all nodes, I want to allow “Plain Text Password” and “domain user credentials” and specific nodes I want domain joined. Therefore, we will write config data as –

$ConfigData = @{
    AllNodes = @(
        @{
            NodeName = "*"
            PSDscAllowPlainTextPassword = $True
            PSDscAllowDomainUser = $true
           
        }
        @{
            NodeName = "DomainJoined"
        }
    )
}

This configuration data I will need use to compile my DSC script.

Compile the DSC configuration

The domain join DSC script has been added to Azure automation account and now it is time to compile it so that .MOF file will be generated on Azure Pull Server. Once .MOF is generated all DSC nodes added to automation account receives configuration from the same .MOF file. If you open the DSC configuration, you will observe that “Compile” button is available at top in the portal itself.
However as of today, if you have DSC script and you wish to get compiled using Configuration data then PowerShell is only option. If you compile DSC script with config data using portal, you will receive errors. So, we will write compilation script and pass above mentioned configuration data, to compile DSC script present in Azure Automation, from local machine.
To compile DSC script present in Azure Automation account from your laptop, you will need credentials. This is where Service Principal helps. When we create Azure Automation account, an Azure Active Directory service principal automatically gets added to your Azure AD tenant under which current subscription is present. To verify, just go to Azure AD on Azure Portal and then select “App Registrations”. You will see the automation account added as web app.



Every app registered in Azure AD has application Id – this is your username.



Every app registered in Azure Ad has secret key – this is your password.



Specify duration as per your choice and give logical name to key as “KeyForDSC” and then click Save. This will automatically generate key. Secret key is visible only once, after which it won’t be visible on the portal. So make sure you get it stored in nice place instead of desktop for future use.
Azure AD tenant has unique id – this is your tenant id.



We need use Login-AzureRmAccount command and use above credentials to start a compilation job for DSC script in Azure Automation account from local machine. The complete script of compilation job with config data and service principal credentials is as below –
$ConfigData = @{
    AllNodes = @(
        @{
            NodeName = "*"
            PSDscAllowPlainTextPassword = $True
            PSDscAllowDomainUser = $true
           
        }
        @{
            NodeName = "DomainJoined"
        }
    )
}

$secpasswd = ConvertTo-SecureString "YourSecretKey" -AsPlainText -Force
$mycreds = New-Object System.Management.Automation.PSCredential ("YourApplicationId", $secpasswd)
Login-AzureRmAccount -ServicePrincipal -Tenant "YourADTenantId" -Credential $mycreds

$compilationJob = Start-AzureRmAutomationDscCompilationJob -ResourceGroupName 'YourResourceGroupName' -AutomationAccountName 'YourAutomationAccountName' -ConfigurationName 'DomainJoinConfiguration' -ConfigurationData $ConfigData
$compilationJob

Run above command and compilation job will start. Once completed, you can view the job status as below –



This completes the configuration DSC domain join in Azure Automation account.

Password in Plain Text

In both, DSC and compilation script above we used password in plain text.

The best way to protect the password is to use certificate based encryption. If you need details around the same then refer - https://blogs.technet.microsoft.com/ashleymcglone/2015/12/18/using-credentials-with-psdscallowplaintextpassword-and-psdscallowdomainuser-in-powershell-dsc-configuration-data/.
However, even if you keep the password in simple text in script it is fine. Because, the .MOF file generated from compilation of DSC script in Azure is always encrypted. But Automation doesn’t know that Azure is going to keep the entire .MOF file encrypted and it throws error. Therefore, we must explicitly specify the force keyword in PowerShell script. So, if you are sure that no untrusted admin will access automation account from azure portal then you can always keep the credentials in plain text. Choice is yours!

Add DSC node to make domain join – finally!

On the Azure portal, under automation account click on DSC Nodes -> Add Azure VM -> Virtual Machines. This will list all VMs present in the azure subscription. Select the VMs of your choice to get them domain joined.
Under Registration tab, Select Node configuration name as “DomainJoinConfiguration.DomainJoined” and provide other information as shown below.



After a while, selected node will display the information as Compliant and VM will show as domain joined.

Bonus Tip - To join VM to specific OU

This is simple. In the main DSC script you will need to provide your OU information if you want all VMs to be joined to specific OU. Example below -
xDSCDomainjoin JoinDomain
{
Domain = $Domain
Credential = $Credential # Credential to join to domain
JoinOU = "CN=Computers,DC=someplace,DC=kunal,DC=gov,DC=au"
}
Sometimes you may face errors related to domain credentials in log generated out of DSC job. In such case instead of specifying user@domain you can also specify domain\user format for credentials in DSC script.


Conclusion

I hope this post have helped you to save you from very repetitive tasks of domain joining a machine.
Please provide your valuable comments. Good news is its free!!

Keep Automating!!

Saturday, May 30, 2015

Export to Excel Microsoft Azure VM List


Introduction


As of today there is no way on full management portal ( http://manage.windowsazure.com ) or on new Azure portal (http://portal.azure.com) to export the list of Azure Virtual Machines to Excel. This blog talks about the same. Actually this is very keen requirement and I have seen lot of people in search of this functionality. This blog is an attempt to simplify their life a bit.
The advantage of Azure VM list export to excel is that you have this excel maintained in TFS or SharePoint to track the Azure environment resources. This simplifies the Azure resource management drastically when the number of VM’s present in an Azure subscription is in hundreds. I have seen my client’s Azure subscription wherein the number of VM’s was 120 and was not having user friendly names. Then every day they were facing challenge to understand the purpose of VM, who is the owner of it, till how long we wish to maintain it, which all need to be shutdown everyday in the late night and to be started everyday in the morning and so on. So I suggested them to have this tracked in an excel sheet. They liked the idea but again came back to me stating that, writing so many VM’s list manually in excel sheet is so boring. Can you please export these Azure VM’s list to excel and provide to us K.
I was thinking there has to be Export To Excel button on Azure portal but unfortunately there is none. So, if you need your Azure VM list to export to excel then the only answer is to write Azure Powershell commands and get it exported as .csv file.
The below is the powershell commands that cane provide you Azure VM list exported as CSV which you can then manually save as Excel sheet. So here we go!

Applicable Technology Stack

  1. Azure subscription with at least 1 Azure  VM’s provisioned in it.

Setting up the connection

Open the Powershell ISE as an administrator as shown below –

First run the command -
Add-AzureAccount

This open a pop up and asks you to enter the credentials of Azure subscription. Please enter subscription live Id and password. This will make current powershell window connected to Azure.

Setting up Subscription of Azure


It may be possible that one live ID account may have multiple azure subscription associated to it. Therefore we need to first select the subscription of Azure from which we wish to export the VM list by using below command –
select-azuresubscription -SubscriptionName "Your Azure Subscription Name Goes Here"

Enter your subscription name in above command. The subscription name can be found from management portal as shown below –


Get Azure VM’s and export the list


First let’s declare an array to hold the list of Azure VM’s.
$results = @() 

Then retrieve the list of VM using below command –
$vms = Get-AzureVM 

The variable $vms consist of all VM’s present in the selected subscription. Therefore we need to iterate over this list to retrieve important information from Azure VM’s and at the end append it to array we defined above.
Once added to array, simply export to Azure with pipe as below –
$results | export-csv -Path "Path to CSV goes here. Example, c:\AzureVMInfo.csv"

So the entire source code I have written in a function as below –
Function Export-AzureVMsToCSV
{
    $results = @()
    $vms = Get-AzureVM      

    foreach ($vm in $vms)
    {
        $details = @{           
                HostName                  = $vm.HostName
                CloudServiceName    = $vm.DNSName
                Name                          = $vm.Name
                VirtualNetworkName = $vm.VirtualNetworkName
                IPAddress                   = $vm.IpAddress
                PublicIPAddress         = $vm.PublicIPAddress
        } 
        $results += New-Object PSObject -Property $details      
    }   
    $results | export-csv -Path "Path to CSV goes here. Example, c:\AzureVMInfo.csv"
    Write-Output "export succeeded"
}
Export-AzureVMsToCSV

This way the .csv file will get created at the specified location with all details. Then you can save it as excel manually by opening the .csv file in excel sheet itself.

Source Code

Entire source code of Azure VM export to excel is available on below location –
https://gallery.technet.microsoft.com/Export-to-excel-Azure-19930e31
Hope this helps.
Cheers…

Friday, January 16, 2015

Azure Blob Snapshots using REST Api and Client Library


My article of Azure blob snapshots deep dive has been published. The article outlines various azure blob snapshots operations using client library and REST Api. The full source code is also available at bottom to download from GitHub. Here is the link –
http://www.dotnetcurry.com/showarticle.aspx?ID=1072

Cheers...
Happy Snapshotting!!

Saturday, October 18, 2014

Multiple web applications in single cloud service web role with multiple SSL certificates and custom domains over https using Server Name Indication


I know the title of this post is quite big and hard to understand in first go. But I wanted to highlight the important aspect I am going to touch in the article and hence such a long name.
Ok, so here in this post I am going to demo how you can host multiple web applications in single cloud service. I know you may say that there are numerous posts on the same topic what is different in here? Yes the difference is there. You may find that most of the posts talks about hosting multiple web sites in single cloud service web role but they do not focus on running them on HTTPS and with custom domain name. Exactly this is what I am explaining in this post.
So in essence, we will see that, how can we host multiple web applications in single cloud service web role. Then we will see how the same applications can be hosted on HTTPS using SSL certificates in single cloud service web role. After that we will extend the talk to hosting same apps with custom domains using new feature of IIS 8.0 called as Server Name Indication (SNI).
Applicable software stack – VS 2013 and above, Azure SDK 2.3 and above, IIS 8.0 and above. [IIS 7 not supported].
Create required project in Visual Studio
This part is pretty straightforward implementation. To start with I created a simple cloud service project in VS2013 and added one web role in it. I renamed the web role to ManegementAppWebRole as shown below –

I changed the default.aspx UI text to just make it identify as Management App. Now I added another ASP.NET web project in the same solution. This will be my survey app. In this also I changed the default.aspx text to just make it identify as Survey app. So in all my projects structure is as follows –

Hosting multiple web applications in the same cloud service web role
Okay, so here I am representing a way by which you can have two web sites hosted in the same cloud service web role. In my case one of the applications is Management app web role itself and another is Survey web application which is not the part of cloud service. So I will run the survey app as part of Management app web role. To configure this full IIS capability can be leveraged. We have a special file in cloud service project know ServiceDefinition.csdef file. My current structure of .csdef file is as shown below –


The <Sites> tag is the key to make this possible. So I just added another site in .csdef file and provided the path of Survey web application. And provided the actual physical path of my survey app. Most importantly if you observe I have provided the host header for my survey app as “surveyapp.kunal.com”. This is the main differentiator of the survey app running in the Management app web role. IIS do not allow to have more than one site listening on the same port without adding host header. We need to provide host header because both of our applications will be running on the same Endpoint port which is 80. Therefore there has to be some way by which IIS can understand the request of user need to be served by which application. Therefore we need host header. The complete structure of my .csdef file is as shown below –

Now in reality my host header site “surveyapp.kunal.com” do not exists. Therefore to make survey app resolve to this URL I need to update “hosts” file of my machine. Hosts file is located at C:\Windows\System32\drivers\etc\hosts. I copied the file to desktop and opened in notepad and added an entry as highlighted below –
Then from desktop I replaced / copy-pasted the hosts file back to its location.
Now let’s run the project in debug mode by pressing F5 and it open the browser with management app with url similar to below – http://127.0.0.1:60123/ Here 60123 is port opened by my emulator, this will different in your case. Open another browser and cope the same url of management app and just replace 127.0.0.1 by our host header name i.e. surveyapp.kunal.com. So final url in new browser window will be - http://surveyapp.kunal.com:60123/and bingo!! I see my management app and survey app running on the same port as shown below –

Let’s run the same implementation in Full IIS and Full emulator mode. This can be configured from the cloud service project properties. To open the properties of cloud service project, right click on Cloud service project and select Properties.
Note – With Azure SDK 2.4 FULL Emulator has been deprecated; however I guess you should be able to run the Full IIS with Emulator Express. This is what I think!!
Ok, now let’s run the project by pressing Ctrl+F5 [DO NOT RUN in DEBUG MOE!!] and here is what I see in browser and IIS –

As you can see, host header has been applied for my Survey Application. This is how we can run multiple web application in single cloud service web role.
Advantage – You are running the tow different web apps in the same IIS. In other words you are running two different web applications in same instance of Azure role and hence you save money. In case of 2 different roles for 2 different web app, you will be charged for role VM instances of 2 applications. In this case you are charged for only one role VM instances.
Now in next part we will see how we can run these web applications over HTTPS and custom domain. This means, I have custom domain mapped for survey as “surveyapp.kunal.com” and now we will have custom domain name updated in hosts file for Management app as “managementapp.kunal.com” plus to run them on https and certificates.