How the different DNAs of Amazon, Microsoft and Google influence their Cloud Platforms.

Posted on by

Disclaimer: This is an opinionated post. The views and platitudes are solely based on my own experience and observations.

AAG – AWS, Azure & GCP

AWS, Azure and GCP are respectively from Amazon, Microsoft, and Google. These companies have different roots, values strengths and weaknesses. Each of them has a different DNA, which influences their cloud services in diverse ways. This article shows a completely different perspective about how the different DNAs of these organizations have been shaping their cloud market.  

AWS – The Retail DNA

AWS has the retail DNA from the roots of Amazon’s e-commerce business culture. Few notable key traits of the retail DNA are shipping fast to be the first, more focus on volume than margins and packaging under own brands known as private labeling.

Amazon launched AWS in 2006. Early adapters and open-source folks went ahead with AWS, this includes many current successful startups who were catching up during 2008-2013. Although Microsoft launched Azure later that period in 2011, it was less matured, and Microsoft did not have a good repo with open-source communities then. Being the first to market without a serious competition, AWS took the whole advantage of the situation during that period.

AWS follows a continuous innovation cycle and keeps on releasing new services although those services are either less popular or only useful to a smaller set of customers. AWS does this to be the first in the market, not worrying about the bottom-line.

Another interesting trait of the retail DNA is Private Labeling. Private labeling is a business technique used by retail players to package common goods from suppliers under their own labels with some value additions. AWS uses this technique very cleverly. AWS has an inherent weakness of not having any established software or operating systems of its own. This does not play well for AWS when it comes to cloud lock-in or giving generous discounts to the customers on software licenses. However, using private labeling AWS has been successfully battling this challenge by creating its own services. Few examples are Aurora DB which is a private label of MySQL/Postgres and Redshift is another successful example.

Azure – The Modern Enterprise DNA

Azure has the DNA of a modern enterprise. Modern Enterprise DNA has old traits like bottom line focus, partner ecosystem and speaking the corporate lingo combined with modern traits such as innovation, openness, and platform strategy.

Azure is not a laggard when it comes to innovations, Azure has its own share of innovative services with more focus on developer productivity and enterprise adaption. Azure Active Directory, Azure Cosmos Database, Azure Functions and Azure Lighthouse are few of those several enterprise-focused innovative services.

Generally, Azure targets its innovations at stable markets where they anticipate greater adaption, they do not invest much on niche market areas just to appear cool. This may be because of the traditional bottom-line focused business orientation. Because of this trait, sometimes we can notice that Azure terminates few services at their beta stage without releasing in General Availability, thus focusing on stable high reach bottom-line focused innovations over diversity of the service portfolio.

Having a rich partner ecosystem is another key strength of Microsoft. This has given an unbeatable position for Microsoft in hybrid cloud market with its Azure Stack suite. Azure Stack is a portfolio of products that extends the Azure capabilities to any environment. It has three products Azure Stack Edge, Azure Stack HCI and Azure Stack Hub. In other terms, Azure Stack is Azure in different versions, loaded in different hardware and bundled together for customers having different hybrid cloud demands. This is only possible by Microsoft because of its long-standing partner ecosystem and OEM partner network.

GCP – Internet Services DNA

GCP has the DNA of an Internet services company; in fact, there is no surprise as it is coming from Google. Google leads the Internet based consumer services; we all use Google services in our day-to-day life. Internet services DNA prioritizes individual services over a whole platform, and it prioritizes B2C over B2B.

GCP is the third largest cloud provider by revenue, but the gap between GCP and Azure is big. Also, GCP has a serious competition from Ali Cloud.

GCP has all the required foundational building blocks of a modern cloud, but it lacks the rich portfolio of services what AWS or Azure has. GCP tries to sell the same thing under different packaging, one example – API management service is listed as ‘New Business Channels using APIs’ and ‘Unlocking Legacy Applications using APIs’. Those are two different use cases of the same product, but not two different services. Though some may debate, this is an approach to attract customers with two different needs, other cloud providers do not do the same trick under their list of products.

Google is a successful Internet services company; Google should have been the leader in cloud computing. Ironically, it did not happen because Google did not believe in enterprise businesses. They were so focused on Internet based services and generating revenue by content advertisements. Individual users were more important than big businesses. When they realized big corporates are the big customers for the cloud business it was bit too late, and they had to bring the leadership from outside to get that thinking.

Google’s Internet service DNA has made GCP fragmented, the perception about GCP as one solid platform is vastly missing. Most of us use GCP services without much attention to the whole platform. We use Google Maps in applications, Firebase has become a necessity for mobile development, we use Google search APIs, but we see them as individual services, not as single cloud platform. The single platform thinking is essential to win the enterprise customers, not having such perception is a major downside of GCP.

However, it is not all bad for GCP, amongst these odds Google seems happy with what they are doing. They are showing upward trend in the revenue, and recently won few notable enterprise customers.

Twitter V2 : Twitter with Elon Musk

Posted on by

It is all over the Internet that Elon Musk is on a mission to buy Twitter. Internet is going crazy about this. Part of it because Elon is doing it, anything Elon is involved get augmented on the Internet for some reason. I thought about what would be the case if Elon is successful this deal and how it would change Twitter.

I ain’t a fanatic follower of Elon. If I have to say one thing I like about Elon, it is his ability to gather great engineering teams and relentlessly push to achieve the great things. I have listed here few things, I would love to see in Twitter, hoping Elon would do these.

Inherent authenticity should be strengthened with Web 3 features

Twitter was started as a micro-blogging platform but quickly earned its place for legitimacy. When someone tweets, it is often considered as a source of statement. Tweets cannot be edited, this character gives a strong essence of authenticity to Twitter. Twitter can augment this with the use of Web3 technologies. Web3 is interpreted in many ways, but decentralized and verifiable characteristic is the focus here. Tweets can be stored / backed by a verifiable cryptographic platform, which will increase authenticity beyond a single entity [Twitter itself] controlling it.

Bots should be regulated.

Twitter is infested with bots. Getting rid all the bots is not easy, it also has implications to some existing platforms and business models. Having right checks and balances and regulating the bots is the possible successful path. Validating the purpose of a bot, allowing bots to have limited reach and allowing them to earn the trust to continuously expand their reach, identifying bots with a different flag, identifying and validating the real entities behind the bots, etc. are few of the many ways to regulate the bots.

New Business Models

Twitter has a weak revenue compared to other social media platforms. Major social media platforms are focused on content economy. Twitter does not have a content based economic model, so it is almost impossible for Twitter to get substantial revenue from content advertisements.

But Twitter is known for its legitimacy; Twitter should validate all entities behind each every Twitter account. This is easy considering the advancements in AI. If you feel I’m talking too dreamy, consider Uber does this with a greater success rate. When the accounts are verified, it opens several opportunities.

  1. Twitter becomes the only verified IAM system at such scale on the Internet, verified and established with Web3 technologies. (Panic button ON for FB and Apple ID). When an identity is verified, many services can leverage it.
  2. Combining this globally verifiable IAM model with metaverse elements like verified metaverse beings opens many other business models in metaverse. A digital twin of a verified person can hold a verified presence in metaverse.
  3. Subscription charges for certain Twitter accounts. But this should be a flat small fee to keep the power balanced and not coupled with any promotional benefits.
  4. Twitter is already testing few features of Web3. Each tweet of a verified account (as we already discussed how all accounts can be verified) linked with a decentralized platform will automatically become an NFT.

Putting it all together – Say you have a Twitter account, you have your username and password. First thing, Twitter would allow you to verify yourself by submitting with the required information. Then you become a verified user (this does not mean you have to be celebrity) Once verified, Twitter can issue you a verifiable Decentralized Identifier (DID). These identities will become a platform neutral (Hopefully this can be achieved with liberal and innovative culture Elon would bring in) decentralized identities for the users. Now, users can navigate across other digital platforms including many metaverse options, with these verified decentralized identities. This will enable users to create authentic digital presence. Digital service providers benefit from linking the authentic digital beings with the real humans enriching their services. At the heart of this, users will have the full control over their identities and they can decide whether to trust or not trust their digital interactions. If this verifiable decentralized identities become a reality then even future elections will use the Twitter backed (but not controlled) decentralized identities.

Once Twitter was one of the revolutionary and very forward-thinking company. Vine and Periscope were good examples of this. However, they are also good examples of how such forward thinking ideas could end up in garbage without a strong visionary leadership. Elon has the greater talent in building great engineering teams. Personally, I like Twitter very much and love to see its future with Elon.

Making of Aventude Calendar 2020.

Posted on by

Ok, this is another brushing about Aventude, but this time it’s not about a customer case, it’s about the calendar we produced for the year 2020. First, when the idea was proposed; it was little out of the traditional way, because technology companies do not create calendars, they would instead create a notebook or nothing at all.

But some felt, let’s do it for some fun and creativity – not to mention I was one of them. The idea of using animals was the first idea, and all agreed, so no much noise in deciding the theme of the calendar. The challenge was, we need to map the technological concepts with nature. We wanted to take a different approach there; we wanted to bring the technology in a more explanatory way, rather than using buzzwords directly.

Also, we wanted to check the quality of the print, images, fonts etc. with one sample. So without spending much time we took one sentence from our corporate slide deck ‘Speed & Quality at Scale’ – Cheetah came in, and we needed a cheetah that resembles both quality and a hunting speed. We reviewed the morphing method eight times as it should be natural as possible without any hard finishes and also colour fade should not disturb the calendar view. Last part we wanted to bring the tech feel from the photos, with neural connection kind of a mesh. The mesh should be based on triangles, and three shades of blue are needed to create a beautiful pattern, we reviewed many combinations of blue on-screen and on print. At this time, the entire concept was not even started, but reviewed the execution viability more than twenty times.

First draft took around two months to get a final cut; I did not want to spend much time and money without seeing it in print, Because if the print is not right, then it will be a wasteful effort. First sample was good and fine to go ahead.

Now, we need 12 concepts. All of us fell in love with the Cheetah, it was amazing on the big screen. So, we decided to keep him, so we needed 11 more. We started with buzzwords, and came up with the lovely and elegant phrases to bring the inner meaing.

MonthPhrases
JanuaryQuality & Speed at scale
FebruaryCollaboration across Geographies
MarchCost Effective Architecture
AprilThree Lane App Modernization
MayAssimilated Engineering with DevOps
JuneData Driven Decisions
JulySimple & Elegant User Experience
AugustEffortless & Powerful Serverless Architecture
SeptemberComposable Service Architecture
OctoberDemocratic Authority & Blockchain
NovemberRationale Intelligence
DecemberUnified Experience & Digital Convergence

Now we have to find the right images. We described what kind of an image is needed, rather than saying that and this. Some are very specific.

We were adamant on we need a peacock, but it is not that all fancy with a fully opened feather (called train). We need a good healthy looking one, should be proud but pure, should resemble elegance and not overly showing-off. I’m pretty sure the design team should have thought, they were stuck with some retards. After 28 reviews, we got the right one.  Some image explanations are too crazy to write here.

However, some were quite straight and quick, ‘Cost-Effective Architecture’ – with one review it came perfect, and this one is my personal favourite.

After completing all the images, we thought ok now it’s time to distribute, but our CEO wanted a nice packaging – Indeed Yes !. An excellent product needs a beautiful packaging.

Now all set, and it went out, we received excellent comments. I thought, little scared that people would think Aventude as a calendar printing company. Lol. The most remarkable comment was from the print agent, as they wanted to use this for their portfolio and the design team was happy they told they hadn’t done a thoughtful design like this.

It was an excellent and an extraordinary effort from them, and no one would have grabbed the idea better than Pixolines. I appreciate and recommend them.

So what now? Are we working on such a thing for 2021? – the answer is No. We may work on something, but it is too early to decide on anything, but it will not be a calendar.

Construction & Interior Design in post COVID19 – Simulations & AI

Posted on by

We are living in a very unusual time, in fact, personally, 2020 is the most challenging year thus far,  and I see the reflections of it in my business and personal life. But times are filled with transformational opportunities rather than allowing us to sharpen the same old knife again and again.

At Aventude Spark-E, we are working with some interesting social distancing induced business cases, and one aspect exciting to me is building architecture & social distancing. It was a request from one customer to obtain technical advisory on how to augment the existing evacuation planning simulations to map social distancing. If you haven’t heard about evacuation planning simulations, simple googling will help, it is a well-established agent-based simulation to study and aid evacuation planning at an event of a catastrophe.

We started it with matching eye-contact based simulation (I was surprised when I saw it first), and it seemed to work well. However, the critical issue is, these simulations are expensive and often loaded as part of pricy software. Also, this software need specialized hardware and processing. Those reasons didn’t map well when we did cost curve analysis for a SaaS application.

Either we have to reduce the cost of the implementation or augment the problem statement to attract the investors and expand the audience. Second option seemed more feasible than first, but how to make it to the mass audience. An idea came across the table, why don’t we make it a standard and we will put social distancing index for each building, that every building has to do it and qualify this index.

An engineering simulation soon became a standardization business – in the back of my head I was thinking, ok, this is how standards are born lol. We gave that task to someone who’s specialized in that area, and started thinking about how to bring more crispy use cases.

At this stage, we were working mostly at the conceptual level or thought leadership level as our PR team prefers that way (wink). Whether the social distancing index would fly or not, the requirement to modify existing buildings and their interiors is a fascinating use case. Most businesses are facing a struggle on how to bring the customer back; it is not enough for them to show they cleaning shoes and tables every hour. Something has to structurally convincing for people to feel safe, because we are fighting against an invisible enemy.

We did some R&D with Revit with a structural engineering designer from customer side, who helped us to do standard simulations and interior basics. The eye contact simulation which already exist, we thought to tap into it and see how things can work together.

The prototype, seemed to be working well.

  • Revit Python SDK is used to study the existing CAD drawing of a building structure
  • It plays a simulation in encode base to identify the eye-contact rate at a given occupancy rate – there are suggestions to use ray-tracing simulation and lighting as well, but we haven’t tried it yet.
  • Revit layer to suggest interior changes to reduce the eye contact rate, which is mapped to the social distancing index.

#3 is challenging but doable; the real challenge is suggesting a building architecture with meaning and taste. Say if you’re a coffee shop, the algorithm should know what are the possible things to put in and re-arrange things in a way that are relevant to your business. It is entirely different from modelling a library. This is the super end goal, but will not be part of the initial release, or it may be available as a preview feature for a particular segment of buildings.

Leaving the details, this is a compelling case in terms of how we are trying to address a creative industry and applying AI to augment it. At this level, a complete AI would be very expensive (or a better way to put it; we still do not know how to make it cost-effective). Now the model suggests the structural elements. These models are more structural that they do not possess aesthetic value; that’s where human creativity and emotion play a role. It is not a real-time human to AI interaction; the baby steps are in more of a guiding mechanism for the designers.

I love to see an AI, that re-models an interior of an existing building, we send a drone or something to capture the building model, and it suggests changes with minimal investment with the current stuff adhering to the preferred choices of interior design.

Apple Exposure Notification API

Posted on by

Of course, I do not prefer much to write about COVID-19, but enticement on the technology does not leave me silent either. In the last post, I managed to cover a holistic view of contact tracing apps, especially how to separate the PII data and analytical.

This week Apple announced its new update iOS 13.5 beta 2, usually I do not go for with the beta updates, but this is a particular case and downloaded the beta with the beta profile.

Little fascination is Apple has taken a similar step, as explained in my post. Fully anonymized and random Ids. Apart from that, these are things to note :

  • Any app to use the Exposure Notification APIs, the app publisher should prove their identity to be an authorized government entity; this will be a cumbersome step to pass, as I understand.
  • Though Exposure Notification API is available in the update, the user cannot switch on the feature without an authorized app installed in the device.

Refer to the images below.


I made this short post to highlight two things.

  • Independent entities who are developing the contact tracing apps, should consider this new update and the acceptance criteria. Since the new Exposure Notification API has been released, Apple may not accept the standard Bluetooth tracing apps like the ones we saw in the last post.
  • The Contact tracing app developers, should pass a clearance from the respective governments to get the apps approved in the App Store

If you’re developing such an app, it’s high time to consider to use the Exposure Notification APIs.

Contact Tracing Apps – A holistic perspective

Posted on by

Contact Tracing Apps are one of the most argued topics these days. Several countries are trying to implement contact tracing apps. Google & Apple announced a joint partnership in enabling contact tracing; it is a two-step approach – first it will be released as an interoperability API, later as a platform level functionality. At the same time, countries like China and Singapore have implemented contact tracing apps including location-based services, this is proven to be effective, compared to Bluetooth based tracing. However, location-based tracing is not widely accepted due to the obvious privacy concerns.

Bluetooth based contact tracing

These applications use Bluetooth to detect who’s around you. Gathered information is then processed either in real time or based on an action. In Sri Lanka there are several projects emerging from different individual developers. Also, there are entities who are trying to implement this solution for the government. We were asked to provide clarity and some working building blocks, to understand the internals of a typical tracing app. This post contains some of the observations and concerns of a contact tracing app from a general perspective.

These kinds of apps trigger concerns of data privacy and related issues (more on this below), but first thing came to my mind was how to do this in iPhone, as iPhone has restrictions in unpaired Bluetooth communication – it requires the app to be in foreground for a successful handshake – you can read more about the limitation from this link

After a quick Internet research, we understood, prevailing COVID tracing apps do have the foreground limitation in iPhone. Also, we came across this app TraceCovid : Fight COVID-19 Together, from the health department of Abu Dhabi.

Here are the screen shots of the app in an iPhone (click to enlarge the images). It is obvious, the app should run in foreground to function properly.

Minimal PII Footprint Implementation

Second concern is data, mainly the PII – Personally, Identifiable Information. PII has a broader data coverage, addition to the obvious data like phone number, email, name, IP address etc.

PII classification and severity vary broadly, which makes it hard to comprehend at times. Couple of interesting examples of PII are, In EU under GDPR, a drawing of a child is a PII as it may reveal the social and environmental impact of a kid’s surrounding. An advertisement put out to sell a car is a PII – not only because you forgot to mask the number plate, because the selling price can be used to inference the financial status of the person at a given time.

With that, note, let us look at how we can implement a contact tracing app with minimal PII footprint. In fact, we can have a contact tracing app with zero PII stored in the systems. Initial validations require a phone number or an email.

User installs the app, enters mobile number, receives one-time password and register. The mobile number is not stored in the systems, it is used to send the one-time password and then wiped off. System generates an id (UUID) like a username and the push notification id will be mapped against this id.  For the system UUID is the user, it has no meaningful mapping to the real person of the UUID. The below deck illustrates how the contact tracing can happen in such case.

This is a fair solution in terms of data privacy, as no PII is persisted in the contact tracing app provider’s systems. However, reaching people is challenging as system relies only on push notifications. Storing mobile number removes this obstacle and eases up the process.

In mapping the UUID to the real person, phone number is preferred because phone communication is more effective in reaching a person at emergency over email. Also, with established policies, telecommunication services should provide APIs to related authorities to retrieve more data about a person based on the phone number.

Data related Concerns

An app of this nature, is a natural victim for concerns related to data, some key concerns would be

  • Data privacy – Discussing this limiting to this context, data privacy is about who will access my data and how they will use it. Will they use this to other purposes than tracing the infection. Will it be shared with others? In case of any findings related to me, will it be shared with others? if so with whom and how they will use it? As you see, data privacy is about how the data is used.
  • Data residency – The geographical location where data is stored. Public cloud or a private datacenter. Within the country or outside. Within a geopolitical region or outside. Within a specific standard datacenter or outside.
  • Data handling – This is very important aspect but often missed. This is the most crucial piece of all. This is about the policies and procedures of the authorized stakeholders who handle the data. This includes screening of such individuals/entities, tools and services used to process the data, the ways data will be processed, what are the data protection facilities of used tools and services etc. This a mix of both technical and processes.

Summary

Plotting the effectiveness of tracing and the privacy, we will end like below.

contact tracing app effectivness vs privacy
contact tracing app effectivness vs privacy

Since the tracing is about finding a specific id and its trails, data analytical component does not require meaningful data, it can work on the anonymized data (as described in the slides) and later be mapped to the real data. Or the entire data set can be processed encrypted using homomorphic encryption.

This allows some freedom in the data residency as well. Anonymized data can be kept in public cloud platform leveraging cheap and scalable infrastructure for real time lambda architecture-based analytics and later brought down to be mapped with the meaningful data.

However, Bluetooth tracing remains obstructive in iPhone.

Azure Lighthouse – A Cloud Native Managed Services Model for Service Providers

Posted on by

Recently Azure announced this service called ‘Azure Lighthouse’. It allows managed service providers and customers to manage the tenant access and the delegation from a single point of interface in the Azure Portal itself. With some marketing garnish, I would like to call it as Cloud Native Managed Service Model. Let me take you through the fundamentals of Azure Lighthouse.

Before proceeding further, this post assumes, you’re familiar with AAD concepts like tenants/directories, object ids, service principles, RBAC etc. I have not referenced or elaborated them here.

Before diving in, let’s look at how the existing managed service providers access their customer tenants. Generally, they use either one of the following.

  1. Service Provider access Customer Tenant as a Guest.
  2. Service Provider access Customer Tenant with a customer tenant user account.

Consider this example, Aventude Digital with its Azure tenant looking for a partner to manage our Azure Resources. MassRover is a managed service provider; Aventude Digital reaches MassRover and requests their service. Bob is the support engineer from MassRover with his UPN (bob@massrover.onmicrosoft.com) should gain access to Aventude Digital tenant.

Scenario #1

Bob gets access to Aventude Digital tenant as a Guest user. In this case Aventude Digital administrator Linda should invite Bob to her tenant, with the required RBAC permissions. Once Bob receives the invitation, he can access Aventude Digital directory. When Bob logs in using his own UPN (bob@massrover.onmicrosoft.com), he can see two directories in Azure – MassRover directory where he is a direct member and Aventude Digital directory where he’s a guest user.

Bob can switch between them and access the resources as per the granted permissions and continue his support work. The invitation process is manual and repetitive. Below image shows, how Bob access different tenants, being the Guest user.

aventude guest directories

Scenario #2

Bob gets a user account from Aventude Digital tenant. Aventude Digital administrator creates a user account in their directory for Bob, something like bob_ext@aventudedigital.onmicrosoft.com. Bob must use this user to access Aventude Digital tenant. This becomes a mess when Bob manages many customers, because he has to switch between different tenants using different UPNs and related passwords. Bob ends up maintaining a table of UPNs and passwords for each tenant he works for.

In short, Guest access is commonly used. But still this is an AAD level delegation only. It is manual and when Bob selects different directories the authentication takes place and the experience is not smooth.

How Azure Lighthouse Improves this.

Azure Lighthouse offers service providers a single control plane to view and manage Azure across all their customers with higher automation, scale, and enhanced governance. With Azure Lighthouse, service providers can deliver managed services using comprehensive and robust management tooling built into the Azure platform. This offering can also benefit enterprise IT organizations managing resources across multiple tenants.

At the core of Azure Lighthouse is Azure Delegated Resource Management, on top of this Azure Portal based Cross Tenant Management Experience comes. Addition to this, we can have Extended Scenarios like Market Place & Managed Apps.

Rest of this post covers the technical implementation of Azure Delegated Resource Management and Cross Tenant Management Experience.

Delegated access can be done by two aspects, one is by manually executing the Azure Delegated Resource Management ARM scripts or by installing the published Market Place Managed Service Offering from the customer. In this post will cover the manual approach.

First, as a service provider, we should create the required ARM template to obtain the Azure Delegated Resource Management permissions from the customer tenant. These permissions can be obtained at subscription level or at the resource group level. Service Provider prepares the required ARM template, and this should be executed at the customer subscription

Below is the ARM template and the associated parameter file.

{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"mspOfferName": {
"type": "string",
"metadata": {
"description": "Specify the name of the offer from the Managed Service Provider"
}
},
"mspOfferDescription": {
"type": "string",
"metadata": {
"description": "Name of the Managed Service Provider offering"
}
},
"managedByTenantId": {
"type": "string",
"metadata": {
"description": "Specify the tenant id of the Managed Service Provider"
}
},
"authorizations": {
"type": "array",
"metadata": {
"description": "Specify an array of objects, containing tuples of Azure Active Directory principalId, a Azure roleDefinitionId, and an optional principalIdDisplayName. The roleDefinition specified is granted to the principalId in the provider's Active Directory and the principalIdDisplayName is visible to customers."
}
}
},
"variables": {
"mspRegistrationName": "[guid(parameters('mspOfferName'))]",
"mspAssignmentName": "[guid(parameters('mspOfferName'))]"
},
"resources": [
{
"type": "Microsoft.ManagedServices/registrationDefinitions",
"apiVersion": "2019-06-01",
"name": "[variables('mspRegistrationName')]",
"properties": {
"registrationDefinitionName": "[parameters('mspOfferName')]",
"description": "[parameters('mspOfferDescription')]",
"managedByTenantId": "[parameters('managedByTenantId')]",
"authorizations": "[parameters('authorizations')]"
}
},
{
"type": "Microsoft.ManagedServices/registrationAssignments",
"apiVersion": "2019-06-01",
"name": "[variables('mspAssignmentName')]",
"dependsOn": [
"[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
],
"properties": {
"registrationDefinitionId": "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
}
}
],
"outputs": {
"mspOfferName": {
"type": "string",
"value": "[concat('Managed by', ' ', parameters('mspOfferName'))]"
},
"authorizations": {
"type": "array",
"value": "[parameters('authorizations')]"
}
}
}
view raw azurelighthouse_subscription_arm.json hosted with ❤ by GitHub
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"mspOfferName": {
"value": "Aventude Ops Servive"
},
"mspOfferDescription": {
"value": "Aventude Ops Service for Azure Managed Customers Tier1"
},
"managedByTenantId": {
"value": "261e3bf5-f768-49cc-a8bb-ab7dcc73817c"
},
"authorizations": {
"value": [
{
"principalId": "6665e9a2-e27a-42f0-8ce1-203c03255695",
"principalIdDisplayName": "Individual User",
"roleDefinitionId": "b24988ac-6180-42a0-ab88-20f7382dd24c"
},
{
"principalId": "52f00b53-e404-4b0e-9564-ffb8388702cd",
"principalIdDisplayName": "User Group Id (reccomended)",
"roleDefinitionId": "b24988ac-6180-42a0-ab88-20f7382dd24c"
}
]
}
}
}
view raw parameter.json hosted with ❤ by GitHub

The ARM template expects certain meta data like the managed service offering name, description and mainly the required delegated permissions (as authorizations). These authorizations are AAD principles (users / groups / service principles) paired with the RBAC roles. The values are fed to the ARM template using the corresponding parameter file.

AAD principle Ids can be found in the relevant blades (we need to use the respective Object IDs) and RBAC role IDs can be obtained from this link

Example: Bob’s Object ID in the MassRover (service provider) tenant is – 6665e9a2-e27a-42f0-8ce1-203c03255695 and we’re requesting a Contributor permission for this user. Azure RBAC ID for the Contributor role is – b24988ac-6180-42a0-ab88-20f7382dd24c. (obtained from the above link). This combination along with a name we provide to be displayed makes one authorization delegated access management record as below.

azure lighthouse authorization snippet

We can add many and different authorizations.

parameter file with different authorizations

Once the ARM template and associated parameter file are completed, customer should execute this in their subscription. In order to execute this, a non-guest user from the customer tenant with Owner permissions to the subscription is required.

PS C:\Windows\system32> az deployment create --name AzureLightHouseDeployment1 --location southeastasia --template-file "C:\Users\Thuru\Desktop\light house blog\json\al.json" --parameters "C:\Users\Thuru\Desktop\light house blog\json\alparam.json" –verbose

It takes, some time and the CLI will spit out an output json.

I used two tenants for this testing. One is called MassRover (service provider) and the other one is Aventude Digital (customer). Above script is executed at the Aventude Digital subscription and script was prepared with the parameters from MassRover. (Bob is in the MassRover tenant).

After execution. In the MassRover tenant Lighthouse, under the My Customers section we can see Aventude Digital.

In the  Aventude Digital tenant Lighthouse, under the Service Providers section we can see MassRover.

This explains the basic of Azure Lighthouse, but it has some limitations at this point. One of the key limitations is, if DataBricks is provisioned in a tenant, then Azure Delegated Resource Management fails, and there are some other limitations too.

If you’re a service provider Azure Lighthouse provides a greater visibility by being in the marketplace. This requires additional setup via partner portal. Also, using service principle delegation, service providers can programmatically automate management tasks. Customers can view the Service Providers at one place including the granted access permissions.

In this post I have covered only one path of Azure Lighthouse, (subscription level delegated resource management), Let me know your experience with Azure Lighthouse and any interesting combinations.

Enterprise data life cycle management using Azure Storage

Posted on by

Storage is one critical component in the Enterprise world. Managing data and its life cycle is a crucial element in many aspects, such as optimizing storage usage, managing cost, adhering to the compliance & archival requirements, security and etc.

Primarily data is stored in database systems (relational and non-relational sources) and as files (includes data lake and blobs), addition to that, data resides in other systems like email servers, document systems, file shares, event and messaging pipes, logs, caching systems and etc.

Laying out a comprehensive data strategy for an organization is a complex process. However, in most cases the data lands in a flat storage as the final tail grade destination. So managing the storage and life cycle management is an important task.

Let’s consider a simple backup storage scenario.

A relational data source assume a SQL Server VM, has following backup requirement.

Frequency Backup Type # backups Access Frequency
4 hours Incremental 42 Medium
Daily Full 30 High
Weekly Full 12 High
Monthly Full 12 Low
Semi-Annual Full 6 Very Low
Year Full 8 Very Low

At any given time (assuming a complete 8 years span) there should be 110 backups maintained. Those 110 backups, should be kept in the right storage based on the access frequency and retention period.

Azure Storage provides access tiers which helps us to determine and auto manage the storage requirements.  Azure storage (storage generation v2) let us define life cycle policies at blob level.

The below diagram depicts this

storage tiers

As shown in illustration, there are three access tiers, hot, cool and archive. Hot and Cool access tiers can be set at the storage account level, and archive tier is set at the individual blob level.

We can define life cycle policies, where the blob movement between tiers from hot to archive and all the way to deletion can be automated to match our requirements.

Sample life cycle policy of a blob.


{
"rules": [
"enabled": true,
"name": "yearly backup rule",
"type": "Lifecycle",
"definition": {
"actions": {
"baseBlob": {
"tierToCool": {
"daysAfterModificationGreaterThan": 30
},
"tierToArchive": {
"daysAfterModificationGreaterThan": 60
},
"delete": {
"daysAfterModificationGreaterThan": 370
}
}
},
"filters": {
"blobTypes": [
"blockBlob"
],
"prefixMatch": [
"backups/annual"
]
}
}
}
]
}

view raw

blob life cycle policy.json

hosted with ❤ by GitHub

You can see, under the filters section, we can specify the path, where the rule should be applied. In this way we can have more than one rule for a storage account addressing different paths.

Out of different options in the Azure storage, we should have a standard general purpose V2 storage, in order to get the access tier capability. Standard blob also has the access tier capability. Standard storage is powered by magnetic disks.

Whereas, Premium storage is powered by SSDs but does not offer access tier. Premium storage is intended for the page blobs, like virtual machine disks. Addition to the page blobs, we can use premium storage as blob storage and file shares.

At summary this is the high level view of the available options in Azure Storage.

stroage summary view