GCast

2018-11-13 · open the original ↗

• Logic Apps are cloud workflows built from drag-and-drop shapes. Giard gives three advantages: they run in Azure and inherit its reliability and scalability, they have a graphical designer, and they ship connectors that replace code you would otherwise write, including SQL Server, Oracle, MySQL, Cosmos DB, Mongo, Azure Table Storage, queues, SAP, email, Event Grid, HTTP requests and functions. A logic app is created under Integration in the portal, and its welcome page offers quick-start triggers (Service Bus queue message, timer, HTTP request) and fuller templates.
• Pasting a sample payload generates the HTTP trigger's schema. For a "when an HTTP request is received" trigger, Giard skips typing a schema definition and pastes sample JSON instead; the designer infers ID, first and last name as strings and quantity as an integer, and the types can be edited. The trigger's URL is generated only once the app is saved; if it is unwanted, he says an Azure Functions proxy can front it.
• Table Storage entities need a unique partition-and-row-key combination. The first step is an Azure Table Storage "insert or replace entity" action, so a repeated ID replaces the existing row. Giard hard-codes the partition key ("all") and uses the posted ID as the row key, storing the whole request body as the entity.
• A Condition control acts as an if statement with true and false branches. Giard tests whether quantity is greater than or equal to 100 and, on the true branch only, inserts the same body into a second "big customers" table with the partition key hard-coded to "big". A test post with quantity 100 landed in both tables; one with quantity 90 landed only in the all-customers table.
• Testing needs real posted data and a JSON content-type header. Running the app from the portal with no payload fires the trigger and then errors because the ID cannot be found. Giard posts from Postman with Content-Type: application/json set explicitly, "otherwise it might just think it's just a string of text", and gets back HTTP 202.

2018-11-13 · open the original ↗

• A proxy puts a friendlier URL in front of a back-end service. Giard gives two reasons: to replace an ugly, hard-to-remember URL with a shorter one whose arguments have meaningful names, and to make a service conform to an organization's URL standards. His back end is the default HTTP-triggered function (built in GCast episode 19), whose URL carries /api/ plus the function name and a long code= key in the query string.
• The route template maps the public path onto the back-end URL. Under Proxies in the function app, Giard adds a proxy with the route template greet/name and a back-end URL that appends name=David, which returns the hard-coded name whatever is passed. Changing the template to take a variable in curly braces and passing it into the back-end query string makes greet/name/Jonathan return "Hello, Jonathan".
• Several route variables can be combined into one back-end parameter. With a template of first name slash last name, he builds the back-end name value as first name, a space, then last name, so a two-segment path returns the full name. He calls the result a URL that "probably would be a good standard for my company to use".

2018-11-07 · open the original ↗

• Deployment Center wires a function app to a source repository. It sits under Platform features in the function app and lists the supported sources, including Visual Studio Team Services (which Giard notes is now Azure DevOps but still shows the old name), GitHub, Bitbucket and local Git. He picks GitHub; the Azure account may need authorizing to read the repository.
• Finishing the wizard adds a webhook and runs a first deployment. Giard selects the GitHub account, the repository and, if there are several, the branch (his has only master), then confirms a summary page. Azure adds a webhook to the repository and, after a couple of minutes, pulls the code and deploys it, so the previously empty function app shows the project's one function.
• Every push to the repository then redeploys automatically. Giard adds a second HTTP-triggered function in Visual Studio, commits it locally and pushes to the master branch, and it appears in the function app without further action; he had planned to wait about two minutes but did not need to. He skips testing the new function, saying he ideally should run automated tests first but that this is "a demo for deployment not for functions".

2018-10-18 · open the original ↗

• Visual Studio offers Azure Functions as a project type. Giard creates one in Visual Studio 2017 under File, New Project, and says a missing template means the latest Azure SDK and tooling are not installed. The trigger choices listed are blob insert, Cosmos DB, event hub, HTTP request, queue, service bus and timer. He picks the HTTP trigger, which needs a storage account; for local testing he uses the storage emulator installed with the SDK rather than one in Azure.
• The template's code greets a name or returns a bad request. The boilerplate looks for a name parameter in the query string, then in the request body. It returns HTTP 200 with "Hello, <name>" if it finds one, and HTTP 400 with "Please pass a name on the query string or in the request body" if not. Giard says the code can be changed to do anything, such as write to a database or call another function or web service. The FunctionName decoration sets the function's name, which he changes from "Function1" to his own.
• The function runs and is tested locally before any deployment. Running it compiles and hosts the function locally, and the console window ends with a local URL. In a browser, that URL with no parameter returned the 400; adding ?name=David returned 200 and "Hello David".
• Right-click publishing is quick but not for real use. Giard publishes by right-clicking the function app and choosing Publish, but warns "don't do this in the real world": code should go into source control and continuous integration. The Publish dialog creates a new App Service, or uses an existing one. He gives it a function app name that must be unique, a new resource group, Central US, and a new storage account on default settings.
• A Visual Studio–deployed function cannot be edited in the portal. In the portal it shows only metadata, not code; changing it means editing in Visual Studio and redeploying. Giard thinks that is probably the better model, because it is easier to get the code into source control from Visual Studio. It can still be tested there with Run (a POST, with the name in the body or as a parameter). It can also be called at its Azure URL, which carries a long code Azure appends for security, and it accepts both GET and POST.

2018-10-18 · open the original ↗

• A function app must be created first to hold the functions. Giard calls Azure Functions "serverless" because the server is abstracted away, not absent: apart from choosing Windows or Linux, he never deals with it. In portal.azure.com the function app is under Create a resource, Compute. Its name becomes a <name>.azurewebsites.net URL, so it must be unique. He picks the consumption plan because it charges only when the function runs, which suits code that runs infrequently. Other choices are Central US, a .NET runtime (JavaScript or Java are also offered), a storage account, and optional Application Insights; creation took about 60 seconds.
• The trigger decides when the function's code runs. New functions can be built in Visual Studio, VS Code, another editor, or in the portal; Giard uses the in-portal option here and leaves the others for a later episode. The quick choices are "Webhook + API", which runs on an HTTP GET or POST and makes the function a web service, and a timer that runs every X seconds. Other templates fire on a message dropped on a queue, event hub or service bus, or on a change in a database. He picks the HTTP trigger and keeps the default name, though he says a meaningful name is better.
• The sample C# code returns a greeting or a bad request. It logs information, which Giard notes is one way of debugging. It reads name from the query string or the request body, which it deserializes with Newtonsoft.Json. With a name, it returns HTTP 200 and "Hello, <name>"; without one, HTTP 400 and a message asking for a name.
• The function can be tested in the portal, browser or HTTP tools. The portal's Run button sends a POST with the body pre-filled as name "Azure", and it accepts a query parameter instead. The function URL is the app's azurewebsites.net address plus /api and a long key, which Giard says can be given to users to protect the function. Pasted into a browser, it needs &name= appended because the key is already a query parameter. He says Postman, Fiddler or one's own code work too.

2018-10-17 · open the original ↗

• Azure Data Factory is SSIS-style ETL that runs in the cloud. Giard describes it as a cloud integration tool for extract, transform and load. He compares it to SQL Server Integration Services, which came free with SQL Server but runs on-premises. His demo moves a comma-delimited file from an Azure Blob Storage container named "Imports" into an empty Customers table in Azure SQL Database. The file has four rows of first name, last name, city and state, and he built the table to the same schema, though he says a transform would let them differ.
• Copy Data is a wizard reached from Author & Monitor. Giard creates the factory with the latest version, in the same resource group as the database and storage account to make them easier to manage. He says the location should be close to the data to minimize latency; he uses East US, and creation took about 30 seconds. Author & Monitor opens a new browser tab, where he chooses Copy Data. He picks "run once" over a regular schedule, because he likes to test before scheduling, and says he would later schedule it to pick up a file of each day's new customers.
• The source connector auto-detects the file format. Giard makes a new Azure Blob Storage connection, filtering the long connector list by "Azure", and browses to the container, which he says is analogous to a folder. The wizard guessed the format from the file itself: comma column delimiter and carriage-return/line-feed row delimiter, either of which can be changed. With no header row, it named the columns Prop 0 to Prop 3, and he renames them in the schema.
• The destination needs a connection, a table and a column mapping. Giard connects to the Azure SQL Database with SQL authentication as the server's admin user, and tests the connection before saving. He then picks the target table, which he notes matters once a database has several. The wizard mapped the renamed columns to the table columns in order, which can be changed, and it offers optional pre-copy scripts.
• By default, the first bad row aborts the whole copy. For bad rows, such as a string going into a number column or a value too long for its column, the alternative is to skip them and optionally log them. Giard keeps abort because he wants to know about a problem. In the Monitor tab, the run showed in progress after about six seconds and then succeeded, and the four rows appeared in the table.

2018-10-04 · open the original ↗

• Cosmos DB's other APIs let existing apps move without major rewrites. Giard's case is an application already on MongoDB. Choosing MongoDB as the API at creation lets it move to Cosmos DB for its global distribution, scalability and speed without modifying a lot of its code. The API is chosen per account, and each account gets its own <name>.documents.azure.com REST endpoint.
• Each API needs a separate Cosmos DB account. Giard creates four in one resource group, all in Central US because he is in Chicago: MongoDB, Cassandra, Azure Table, and Gremlin, which he describes as the graph API. He says to place an account near the applications or users. He names each one after its API so he can tell them apart, and all four finished deploying in about five minutes.
• The non-SQL Quick Starts give code samples, not a full app. Unlike the SQL API's downloadable .NET to-do application from the previous episode, the MongoDB Quick Start gives a connection string and snippets. Its fuller samples cover CRUD (create, read, update, delete) in .NET, Node.js, the MongoDB shell, Java and Python, with links on migrating to Cosmos DB. The Table and Cassandra Quick Starts offer .NET, Node.js, Java and Python, plus shell for Cassandra. Gremlin offers .NET and the Gremlin console, and a downloadable Visual Studio application once a container is created.
• Per-API libraries hide the REST calls underneath. Giard says every API is REST under the hood, abstracted away by the libraries for each API. He presents the Quick Start code as something to copy into an application and modify.

2018-10-04 · open the original ↗

• Cosmos DB is fast because it enforces no schema. Giard describes it as a globally distributed, highly scalable NoSQL database that evolved from Azure DocumentDB. "NoSQL" here means the database enforces no schema, relationships or referential integrity. He says that is why it can be very scalable and very fast.
• One database service offers several APIs to ease migration. The API is chosen at creation: SQL, MongoDB, Cassandra, Azure Table or Graph. Giard says code already using one of these can be moved to Cosmos DB more simply for its scalability, global distribution and fault tolerance. The account name becomes a <name>.documents.azure.com endpoint, so it must be unique. He picks the SQL API and Central US, leaves the other defaults, and says creation takes a couple of minutes.
• Quick Start builds a collection and a runnable sample app. Quick Start offers SQL API samples in .NET, .NET Core, Xamarin, Java, Node.js and Python. Choosing .NET Core and "Create Items collection" creates the collection and downloads a zip holding a Visual Studio 2017 solution. It relies on the NuGet package Microsoft.Azure.DocumentDB.Core, which Giard says keeps Cosmos DB's original name. He describes it as an abstraction layer over the REST API that uses syntax very similar to SQL.
• The sample code covers CRUD but hard-codes its credentials. Giard calls the key and URL in the repository code "a very bad practice"; they belong in a config file. The constructor creates the "ToDoList" database and the "Items" collection if they do not exist. Every method is async because each one is a web-service call across the internet. They include GetItemAsync by ID, GetItemsAsync filtered by a boolean expression, and create, update and delete. He suggests using it as a starting point rather than as is, because it is specific to a to-do list.
• Items added in the sample app appear as JSON documents. The sample is an ASP.NET Core web application for a to-do list. Giard adds three items, and they show up in the portal's Data Explorer stored as JSON documents.

2018-09-22 · open the original ↗

• A web app's name must be globally unique because it becomes its URL. Giard creates it from "Create a resource" > Web > Web App; the name becomes the public address http://<name>.azurewebsites.net (his demo is dgtestweb). The app can run on Windows, Linux or a Docker container. Creation took about a minute and produced a sample page at that URL.
• The App Service plan sets both the hosting location and the machine size. Giard says to place it near your users, near any back-end resources the app calls, and near yourself while building; being in Chicago, he picks Central US. The pricing tier buys more or faster processors, more memory and solid state drives; he takes the standard tier.
• Web apps support ASP.NET, PHP, Python, Java and Node. The settings choose which languages are on and their versions, for example Python 2.x or 3.x, along with the default page and authentication or authorization.
• Code can be deployed by FTP, through staging slots, or continuous deployment. Deployment credentials set an FTP username and password. Deployment slots give a staging copy to test in Azure and then swap into production, and several slots can serve different test stages. Continuous integration and deployment can pull from Visual Studio Team Services, OneDrive, a local Git repository or GitHub.
• Giard values that all App Services share one feature set. Web apps, mobile apps, logic apps and web services get the same settings, so, as he puts it, a feature added to one is added to all. Hosting them on the same platform also makes it easier for one to call another and pass identities between them.

2018-09-07 · open the original ↗

• A storage account holds blobs, queues, tables and files under one unique name. Giard creates it at portal.azure.com from "Create a resource" > Storage > Storage account. The name must be unique because it becomes a URL under core.windows.net, and users reach the account through a REST API at that URL. Files are built on top of blobs and emulate a file share for applications.
• The replication choice decides where the extra copies of your data live. Azure always stores several copies. Locally redundant keeps them in one data center on different fault domains, so they share no rack or power supply but sit within about a hundred yards of each other. Giard says paying a little more for geo-redundant storage keeps a copy in another region, such as Texas for a Chicago data center. A third option also makes that copy readable by your users.
• The rest of the creation form is mostly defaults and optional extras. Giard picks a general-purpose account in East US and the standard performance tier rather than premium ("I'm a cheapskate"). The form also sets secure transfer, the subscription and a resource group, which he describes as a logical grouping of resources managed together. Optional settings put the account on a virtual network for secure access from VMs on it, and enable Data Lake storage.
• Automation options export the configuration for repeatable creation. Before creating, the automation options offer an ARM template with its parameters, plus code in CLI, PowerShell, .NET and Ruby. Giard thinks it downloads as a zip file, which can recreate exactly the same storage account.
• The created account's blades manage containers, tables and queues directly. Creation took about a minute. Under blobs he creates a container (the name must be lowercase, a rule he says he always forgets) and can upload files to it. Tables take an access policy, and both tables and queues show a REST endpoint. Further blades cover security, scalability, CORS and encryption, each linking to documentation.

2018-08-28 · open the original ↗

• Encoding a video does not by itself give it captions. Giard creates the Media Service from Create a Resource with a new resource group and a storage account for the media, uploads a video on the Assets blade, and encodes it with the defaults. Encoding runs as a job that moves through scheduled, queued, processing and finished; its length depends on the video's length and size, and the result is a second asset with no captions.
• An Analyze job on the original video generates the caption files. By default it writes captions in three formats, all checked; Giard thinks VTT is the most popular and uses it. The same job can also produce index files for SQL Server full-text search and a keyword file; he keeps the defaults with English as the language. Analysis is also queued, which Giard explains as Azure using resources efficiently for a one-time, non-critical process, and it yields a third, indexed asset.
• Auto-generated captions need checking, and the VTT is plain text. To get a download URL, the indexed asset must first be published with a progressive locator; Giard then downloads the file with curl (curl -o poe.vtt <url>) because his browser could not work out its MIME type. The captions of his reading of Poe's "Annabel Lee" contained mistakes, which he corrects by hand in Notepad before uploading the file as captions to the encoded asset.
• Streaming requires publishing and a running streaming endpoint. After publishing the encoded video, the portal shows the streaming endpoint in red until it is started; Giard notes that with no restrictions added, anyone can view it, and that restrictions can be added later.
• Azure Media Player shows captions only after you add a track. Giard pastes the playback URL, without the http: but keeping the //, into the Azure Media Player demo page; by default there is no captions menu. Under the advanced options he adds a track of type captions or subtitles, with a label (several caption sets are allowed), a language, and a URL at the same location as the .ism manifest but ending in the VTT file's name, after which an English captions menu appears. He says step-by-step instructions are in an August 2018 post on his blog.

2018-08-28 · open the original ↗

• Live streaming runs through a channel on the live streaming blade. Custom create lets you set the encoding type, name, description, auto-start, streaming protocol and IP restrictions; quick create takes the defaults, which gave Giard pass-through encoding and RTMP ingest. He warns that you are charged while a channel is running, so it is worth being diligent about stopping it.
• The channel's ingest URL is where the streaming software points. The channel blade lists a primary and a secondary ingest URL, and either works. The channel came with a live event named "default", already started and published but not streaming until a source is sent to it.
• Separate streaming software sends the stream to the channel. Giard uses Telestream's Wirecast, priced at $449 to $949, on its free trial, which stamps a watermark about every 60 seconds, fine for testing but not for production, in his view; install needed one restart. In Output Settings he chooses an RTMP server as the destination, pastes in the ingest URL, adds a screen-capture source (layers can add a camera or branding), moves the preview to live, and starts the RTMP server.
• Viewing needs the streaming endpoint running, and "live" lags. The stream does not play until the streaming endpoint is started; then Watch in the portal shows it. Giard notes it runs a few seconds behind, since there is always latency going out to the web and back.
• Anyone can watch through Azure Media Player. He pastes the channel's .ism/manifest URL, without the http: prefix, into ampdemo.azureedge.net to share the stream with people outside the portal.

2018-08-28 · open the original ↗

• A media service comes with storage and a streaming endpoint. Giard creates it from Create a resource, under Mobile, giving it a resource group, a region (he advises one close to you or your viewers) and a new storage account, whose name has to be unique. Deployment took under a minute and created the resource group, storage account, media service and streaming endpoint.
• An uploaded video cannot play until it is encoded. The encode dialog offers several formats; Giard keeps the default MP4 and job names. The job goes from queued to processing to finished and encodes into several formats for adaptive streaming, leaving the original and an encoded asset side by side.
• The encoded asset must be published and streaming started. Play stays disabled until the asset is published and the streaming service is started; a checkbox on the publish step starts it too. In his demo this took about three minutes, after which the portal's own player played the clip.
• Sharing outside the portal needs a generic web player. Because viewers may not have Azure access, Giard uses Azure Media Player: he pastes the playback URL, which must contain ism/manifest, minus http:, into the free player at ampdemo.azureedge.net. The player then gives a link to send to people, and code to embed the player in a web page.

2018-08-09 · open the original ↗

• Chocolatey installs Windows software and its dependencies in one line. Giard calls it one of his favorite tools: a free package manager, Windows-only, installed by pasting the command from chocolatey.org into a PowerShell or cmd window, probably with administrator rights.
• choco install takes the package name from the site's package list. Running as administrator, he installs Visual Studio Code with the command shown on its package page. Without -y Chocolatey asks before running each script (twice for this package); -y answers yes to all of them.
• A batch file of installs rebuilds a fresh machine unattended. When he reformats and reinstalls Windows, Giard runs a batch file with one choco install <package> -y line per tool, such as Node.js, an FTP client and a PDF reader, and comes back a couple of hours later to find it all installed.
• The result is the same as a manual install. Command-line tools need a new command window before they are recognized; after that code . launches Visual Studio Code, and it also shows up in the Start menu, as if downloaded and installed by hand with its prerequisites.
• choco uninstall removes the app and its dependencies. Uninstalling Visual Studio Code with -y took it out of the Start menu, and code . was no longer recognized in a new window.

2018-07-29 · open the original ↗

• Azure supports Linux virtual machines extensively, despite the Microsoft name. Giard says that the last time he read, about 30 to 40% of the virtual machines in Azure were Linux. Under Compute in the portal he names Red Hat Enterprise Linux 7.2, Ubuntu 17.10, CentOS and SUSE. Some images are managed by Microsoft and some by third parties, so he tells viewers to check the licensing on each.
• A Linux VM is created the same way as a Windows one. He picks an Ubuntu Server 14.04 LTS image managed by Canonical. He chooses a regular hard drive or a solid-state drive, then public-key or password authentication (he takes password because "passwords are easier"). He creates a new resource group and picks South Central US for low latency from Chicago. The size he selects has 2 CPUs, 8 GB RAM and 4 data disks, at about $81 a month if left running for 30 days straight.
• Optional settings cover redundancy, networking, extensions and cost-saving shutdowns. He points out availability sets for fault tolerance, a shared virtual network or subnet, and extensions such as antivirus and backup, but uses none of them. An automatic daily shutdown at a set time and time zone, with an optional email notice, saves money. Downloadable templates let the same machine be created again from PowerShell, the CLI or .NET.
• Remote access requires opening the inbound SSH port. This setting is required. He opens 22, the default SSH port, and accepts the portal's warning that exposing a port to the Internet is a potential security risk. Deployment took about five minutes.
• Linux connects through an SSH command, not a Remote Desktop file. Clicking Connect on the overview page gives the command ssh followed by user@IP address, where a Windows VM gives a downloadable RDP file. He pastes the command into a command prompt, confirms the prompt about the machine having no certificate, and enters the admin password. From there everything happens at the command line, such as installing software and launching services.

2018-07-28 · open the original ↗

• Azure VMs are portable between the cloud and on-premises. Giard describes a virtual machine as a file that emulates a physical computer, with its memory, disk, software and data. Because Azure uses the same hypervisor technology (Hyper-V) as on-premises virtual machines, he says they move between the cloud and a local machine or network "with relative ease".
• The image gallery offers operating systems and pre-installed software. Under Compute in the portal there are versions of Windows Server, Linux distributions such as Red Hat and Ubuntu, and images with software already installed, such as SQL Server. He picks Windows Server 2016 Datacenter. He then sets a name, a regular or solid-state drive, admin credentials (the password must meet rules shown by hovering the warning icon), a subscription and a new resource group.
• Pick a region for latency; bring or rent a Windows license. He says to place the VM near you, your users and any Azure or web resources it talks to, and picks East US. If you already own a Windows license you can use it. Otherwise Microsoft rents you one and the hourly cost goes up a little.
• You pay only while the machine is running. The listed monthly cost assumes running 24 hours a day for 30 days. Starred sizes are the ones available in the chosen region. The summary page showed about 37.6 cents for 60 minutes, billed by the minute. An auto-shutdown, for example every day at 7 p.m. Central, avoids charges for the hours after that. He also notes availability sets, a shared virtual network or subnet, extensions such as antivirus, and downloadable templates (PowerShell, CLI, .NET) to automate creating the same VM.
• Remote Desktop access requires opening its inbound port. This setting is required. He opens 3389, the default Remote Desktop port, despite the warning about Internet exposure. After about five minutes of deployment, the overview page shows the status, region and public IP, with buttons to reset, stop or connect. Connect downloads an RDP file that opens in the Remote Desktop app. He accepts the warning about the missing certificate because he just created the machine, and notes that the first login is slower while it sets up a profile.

2018-06-28 · open the original ↗

• MySQL is now a built-in Azure service, not a marketplace add-on. Giard says it used to come from the Azure Marketplace at extra cost, or you installed it on a virtual machine yourself. Now you go to Create a resource, then Databases, then Azure Database for MySQL.
• Creating the server takes a few settings, each with a choice. You can start with a blank database or restore one from a backup. You set an admin username and a password that must meet the rules shown on screen. You pick a location: Giard says to choose it by where the consuming applications, managing users and other resources are, and he takes East US. He takes the latest MySQL version and the default pricing tier, General Purpose with 2 vCores, which the portal estimates at about $131 for 30 days running 24 hours a day.
• The portal's Cloud Shell connects to the server from the browser. The Cloud Shell icon opens a Bash shell, where mysql -h <hostname> -u <username> -p connects; -p prompts for the password. The server comes with four databases. He creates a new one, makes it the default with use, then creates a five-column table, inserts rows and selects them back.
• Applications connect using the portal's ready-made connection strings. The Connection strings page lists them for several data-access technologies. You fill in the username, password, database name and host name, all of which appear on the Overview tab.

2018-06-28 · open the original ↗

• Azure SQL Database is SQL Server without managing the server. Giard describes it as built on Microsoft SQL Server, with almost all the features of an on-premises install. The server, the machine and the virtual machine underneath are abstracted away.
• Create the logical SQL server first; its name must be globally unique. Giard prefers to create the server on its own, though both can be created at once. The name has to be unique among Azure SQL servers because it becomes a public URL, <name>.database.windows.net. He sets a login, a password that meets the rules shown on screen, a new resource group and East US. He chooses the location by where the users and consuming applications are, to keep latency low, and notes there may be legal limits on where data can be stored.
• The database starts blank, from sample data, or from a backup. Sample data is for learning SQL Server or for building an application against test data. Giard takes a blank database on the server he just created. There are three pricing tiers, Basic, Standard and Premium, each sized further in DTUs, a measure of processing and memory. Their maximum database sizes are 2 GB, 250 GB and 1 TB, and you can set them lower to cut cost. He takes Standard and the default collation.
• Connection strings and management features come ready-made in the portal. Connection strings are given for ADO.NET, JDBC, ODBC and PHP, with the username and password left out. Auditing, alerts, diagnostics and geo-replication are switched on by setting properties. The database can be managed with the free SQL Server Management Studio, Visual Studio, the Cloud Shell in the browser, or PowerShell on a local machine.

2018-06-05 · open the original ↗

• Databricks makes Spark easier to use on Azure. Giard describes it as built on HDInsight Spark, Microsoft's Hadoop on Azure, with Spark as its general-purpose compute engine. He says Spark "isn't always the easiest thing to use". Databricks adds a layer that makes it simple to deploy jobs, create Jupyter notebooks and auto-scale by setting configuration properties.
• A workspace comes first, then you launch it. He creates the Azure Databricks resource with a name, a resource group, Central US and the cheaper pricing tier, and pins it to the dashboard. It took about two or three minutes. Launch workspace signs him in from cached credentials, and from there he can create notebooks, jobs, clusters, tables and libraries.
• Cluster autoscaling and auto-shutdown control how much you pay. He picks Apache Spark 2.3 and Python 3. With auto-scaling on, which is the default, the cluster adds a worker node when compute runs low, up to a maximum of 8 that he says can be raised to 10. He sets it to shut down after 60 minutes of inactivity. His reason is that you pay for every node for as long as the cluster is up. The cluster took about five minutes to create.
• A Python notebook on the cluster runs pandas and Markdown cells. He attaches a Python notebook to the cluster. He sets a file-name variable to a CSV of forest-fire data, imports pandas, and reads the file into a data frame. df.head shows the first five rows, or a number you pass in, and the last rows can be shown the same way. Because the notebook defaults to Python, a Markdown cell has to start with %md.

2018-06-04 · open the original ↗

• Azure Notebooks hosts Jupyter notebooks in the cloud for free. You sign up at notebooks.azure.com. Get Started lists sample notebooks, which Giard recommends for learning Jupyter, Python and machine learning. A notebook mixes formatted text with executable code and its output.
• Notebooks live in libraries, which can be made public. Under Profile, then Libraries, you create a library with a name and a URL slug under your account. Making it public lets anyone browse your notebooks. A new notebook can be uploaded, copied from a URL, or created blank or with a default language: Python, R or F#. Giard picks Python 3.6. Opening a notebook starts a notebook server, the engine that runs the code and formats the text.
• Variables and imports carry across cells, and cells can be Markdown. Giard sets a file-name variable to a forest-fire CSV hosted at UC Irvine and imports pandas. He runs cells with the run button or Shift-Enter. A later cell reads the file and shows the first five rows by default, the last rows, or a count you pass in. Switching a cell's type from code to Markdown makes it render as formatted text.

2018-06-04 · open the original ↗

• Cloud means third-party hosting that is highly scalable and highly reliable. Giard gives this definition because he says marketers have "hijacked" the term. He groups cloud services into infrastructure, platform and software as a service, while noting that not every service fits neatly into one. On reliability, he says Azure writes multiple synced copies of your data, three by default, with one live. The Azure Service Fabric monitors them and fails over to another copy when the live one goes down.
• Each cloud layer abstracts away more of what you must run. The cloud grew out of virtualization, which turns a whole machine into one portable file. Running your own data center means paying for hardware, security, climate control and patching, none of which is your core business. IaaS hosts your virtual machines, but you still patch their operating systems. PaaS also hides the virtual machine: you ask for a database or a web app and just use it. SaaS, such as Gmail, Outlook.com, Office 365 and Salesforce, leaves the customer nothing to run.
• Start with SaaS and move down only when you hit a limit. Giard frames the layers as a trade between less work and more control. He recommends using an existing service if one does what you need. Where it does not, combine PaaS services with your own code, and fall back to your own virtual machine only when something like installed software or a registry setting forces it. He says this tends to cost less.
• Elasticity means paying for capacity only while demand needs it. At an electronics retailer he once worked for, about 80% of revenue came between Thanksgiving and New Year's. Hardware bought to cover that peak sat mostly idle the rest of the year, and averaging the capacity would have lost sales. In the cloud you go from 2 instances to 20 and back by configuration, or by CPU thresholds that add and remove instances. His other examples are ticket on-sales, a sudden spike from a celebrity mention, and data science's short bursts of heavy compute. He credits the cloud for the field's recent growth.
• Azure bills by the minute and spanned 50 regions in May 2018. Prices are quoted per hour at azure.microsoft.com/pricing but charged per minute. They scale with machine size and rise linearly with instance count. As of May 2018 he counts 50 regions in 140 countries, which he says give near-unlimited capacity, low latency near customers or data, and copies across regions for fault tolerance. A status page shows whether a fault is yours or Azure's. In his web-app demo the name must be unique because it becomes <name>.azurewebsites.net. The app service plan sets location and size, and Standard (one core, 1.75 GB) is estimated at about $74 a month.

2018-06-04 · open the original ↗

• ML Studio turns a trained model into a deployable web service. This builds on the census-income model from the previous episode, which predicts whether a person earns over or under $50,000. Of that episode's two algorithms, which scored nearly the same, Giard picks one. Choosing Set Up Web Service, then Predictive Web Service, creates a new experiment with web-service inputs and outputs. After it runs, Deploy Web Service publishes it to Azure.
• Request-response calls POST JSON and can score several people at once. The documentation gives the URL, the headers, and a JSON body holding an array of inputs, so one call can carry multiple people. It also describes the JSON response and each field's type, and gives sample console-app code in C#, Python and R. A test console can fill in the inputs from the 30% of data held back for testing, and returns the actual and predicted income.
• Batch scoring takes a CSV and three separate calls. First you submit a batch job whose body points to an uploaded CSV, then you start it. A 200 response means only that it started, not that it finished. You then poll the status with a GET, which returns not started, running, failed, canceled or finished. Finished is the value 4. Sample code is given for C#, Python and R.

2018-06-03 · open the original ↗

• ML Studio's drag-on shapes replace machine-learning plumbing code. Giard's point is that steps common to most solutions are shapes you drag on and configure, so your time goes to what is unique: getting data, splitting it into training and test sets, and cleaning it. It is free at studio.azureml.net, or you can use a workspace in an Azure account to get Azure's scalability.
• One-line Python scripts and built-in shapes clean the census data. The built-in Adult Census Income Binary Classification dataset labels income as under or over $50,000. Because 90% of rows have the United States as native country, he filters to those with a one-line Python script, cutting 32,561 rows to 29,170. The script's azureml_main function takes up to two data frames. Clean Missing Data, set to remove the entire row, then leaves 27,504.
• Choosing features takes domain knowledge and avoiding redundant columns. He keeps the numeric years-of-education column over the text education column, since keeping both would double-count education. Using Select Columns he keeps age, years of education, marital status, occupation, race, sex, hours per week and income. A second Python script maps marital status from seven values to married or not married, which splits 47% to 53%.
• Hold back test data and use Microsoft's cheat sheet to pick an algorithm. Split Data defaults to 50/50, but he uses 70% for training and 30% for testing, because a model has to be tested on data it has not seen. Microsoft's Azure Machine Learning algorithm cheat sheet points to two-class classification. He rules out the option it marks as built for more than 100 features and picks Two-Class Averaged Perceptron. Train Model needs income set as the column to predict. Score Model and Evaluate Model show about 81% accuracy, against a coin-flip baseline.
• A second algorithm in the same experiment scored almost the same. Two-Class Logistic Regression, trained and scored on the same split, reached 82% against 81.9% for the perceptron. Giard says the next step would be weighing false negatives against false positives if one costs more, or changing the columns or their weights. His verdict is that this is a strong starting point built with about six lines of code.