Souk's Notes - Unofficial Summaries

2024-01-31 · open the original ↗

• Starlink carries about 42 petabytes of traffic a day. The host cites a PCMag article for the figure (42,000 TB, or 42 million GB, daily), and passes on a Reddit user's calculation converting it into an equivalent number of 3.5-inch floppy disks a day.

2024-01-31 · open the original ↗

• Decompressing and re-encoding beats editing sorted lists in place. Eini says that because the compression runs faster than memcpy and moving data to the CPU is the dominant cost, Corax updates a posting list by unpacking a buffer, scanning it alongside the additions and removals, and re-encoding it; a 4 KB buffer holds about 7,000 document IDs, so the index updates incrementally without Lucene's deferred merge cost. Part of this rests on a machine-generated code file of about 17,000 lines.
• Corax lets users query any combination of fields without compound indexes. He says a traditional compound index (last name, then first name) fixes its structure up front and cannot serve a first-name-only search, so supporting arbitrary queries means a combinatorial explosion of indexes; Corax intersects independent, compressed posting lists per term instead, which he says is largely how Lucene works too.
• Sorting a large result set nearly broke the design. His example is "all published blog posts, sorted by date, top five" over 70 million records in a data set of a few hundred gigabytes: Lucene caches every unique value of the sort field in memory, but Corax cannot assume data stays unchanged, and fetching each date meant random disk reads. Corax now rewrites such queries internally, speculatively scanning the date index from the bottom up and matching against the results so they come back already in descending order.
• Corax shipped with zero client changes, by deliberate rule. Eini says it reached the 10x goal, was released in early October (about three months before recording) and is switched on by one configuration setting; customer feedback is "it's faster", alongside some missing Lucene features and bugs. The no-visible-change rule comes from RavenDB's 2015 break of backward compatibility, which he calls the right technical decision but "really annoying", since he still supports old versions, including users of a Silverlight management interface from 2012–13.
• Infrastructure vendors end up debugging systems nobody remembers. He describes a customer system that threw an unhandled optimistic-concurrency error after three years because two people finally edited the same document, with its authors gone and the operations staff knowing only the process name; he cites Leslie Lamport's line that a distributed system is one where a server you never knew existed breaks yours. For one client who refused source control, he shipped a zip of all sources embedded in each deployed binary.

2024-01-31 · open the original ↗

• Lucene is built for a search engine, not a database. Eini says Lucene, which RavenDB used for about 15 years, assumes large periodic batches merged later, so a small index write takes milliseconds until a merge hits and takes 30 seconds; a database sees steady small writes (he cites 100 a second) and users expect to query a change immediately.
• Lucene's compact file format costs RavenDB twice per query. He says the format dates from the early 2000s, when disk was expensive, and must be decoded into managed objects at query time and then garbage-collected; against his target of one million requests per second, cutting allocations "even a little bit" adds 2–3% to overall performance. He says RavenDB already exceeds 500 queries per second on a Raspberry Pi Zero (quad-core 1 GHz ARM, 512 MB).
• Replacing Lucene had to pay for its risk tenfold. Work on Corax (Latin for raven) began in 2014 but was never fully staffed for years; the bar Eini set was 10 times faster than Lucene for indexing and querying in all common scenarios, without losing Lucene's features.
• An index is conceptually simple: terms mapped to posting lists. On a slide he shows an index as a dictionary of fields to terms to sets of document IDs, queried by intersecting sets; for persistence Corax encodes terms with HOPE (High-speed Order-Preserving Encoder), whose compressed keys sort the same as the originals, letting him fit a 100 GB test data set onto the 512 MB Pi Zero.
• Corax uses 64-bit document IDs, compressed with SIMD in C#. Lucene's int32 IDs cap it at 2 billion records, which RavenDB users hit and had to shard around; to keep 8-byte IDs from bloating posting lists (8 MB per million), Corax uses a published integer-compression algorithm the captions render as "Fast P4", which Eini says took him three weeks to understand after five years of returning to the paper. He implements its SIMD instructions in C# through .NET JIT intrinsics rather than P/Invoke, which he says costs too much at billions of calls a second, and says the code hits hardware memory-bandwidth limits while effectively outrunning memcpy because it works on compressed data.

2024-01-31 · open the original ↗

• An indexer turns mixed documents into one queryable JSON index. Giard sets out the sequence: put documents in a data store, create an Azure AI Search service, add the store as a data source, then run an indexer that converts the content to JSON and builds an index, with optional post-processing such as OCR and image analysis. His inputs are nine coffee-shop reviews from Microsoft Learn, which he saved as Word, text, PDF and image files so the service has to read each format.
• Setup is a blob container and a Basic-tier search service. He uploads the files to a blob container named "reviews" with anonymous access, which also has to be allowed on the storage account. He says real use needs more security than that. He picks the Basic tier, the cheapest paid one, at about $75 a month if it runs all month. He says everything shown also works on the free tier, and higher tiers buy scale at higher cost.
• Built-in AI enrichments extract text, sentiment and image descriptions. On import he extracts content and metadata, keeps the default parsing mode because the files differ in structure, and attaches the free built-in Azure AI services (formerly Cognitive Services). He turns on OCR (merged into a merged-content field), page-level granularity at 5,000 characters, location names, key phrases, sentiment (the reviews carry no star rating), and image tags and captions. He skips people, organizations, language detection and PII because the data has none of them. The enrichments and image projections are saved to a knowledge store in the same storage account.
• Index fields and the indexer schedule are set in the wizard. He marks the returned fields (content, locations, key phrases, sentiment, image tags and captions) as both retrievable and filterable. He runs the indexer once, but it can run hourly, daily or on a custom interval such as every 120 minutes from a later start date, which he suggests for a document store that changes. Among the advanced options he leaves on the encoding checkbox, whose main benefit he says is allowing spaces in key names.
• Queries hit stored JSON, so the analysis runs only once. In the Query Explorer, a query for location Chicago returns JSON with a match score, the content, key phrases and a sentiment label for each review. Seattle reviews are excluded. A search for negative reviews returns repeats because the same text exists in several file formats. The indexer also writes a JSON file per document to blob storage holding the metadata and the OCR and image-analysis output. Giard says the index reads that file rather than re-processing the original document.

2024-01-30 · open the original ↗

• Ubiquiti will make two-factor authentication mandatory on every UI account. It takes effect on July 22, 2024. Accounts that have not set up 2FA by then will be defaulted to email codes, which the host calls the least secure and least convenient option.
• The host prefers TOTP or UI Verify over email codes. TOTP is a rotating six-digit code every 30 seconds from an app such as Google Authenticator or Microsoft Authenticator, or from a hardware key like the YubiKey 5 series he uses himself. UI Verify instead pushes an approve-this-login prompt to the phone. Either one covers every UniFi application: Network, Protect, Access, Talk and Connect.
• 2FA is set up under the account's Security settings. Go to account.ui.com, or open any UniFi mobile app and choose Manage UI Account from the profile picture. Under Security, re-enter the password if asked, then use "add new method" under multi-factor authentication. The host hopes Ubiquiti will add FIDO2 or passkeys later.

2024-01-28 · open the original ↗

• This post includes paid content that is not available publicly. See the original source.

2024-01-27 · open the original ↗

• Everyone has something worth sharing, however junior. Eaton, who has talked with many developer boot camp graduates, says newcomers assume they have nothing to say; he objects to a social media post asking people to stop blogging about SOLID, because a beginner's fresh wording may be what finally makes the principles click for someone who has struggled with them. "Don't gatekeep," he says.
• Write primarily for your future self. His "Things I Learned" blog series (an attempt at daily posts that lasted 15 days, about 20 posts so far) records small fixes such as controlling Spotify from the terminal and an Azure Functions failure on his Mac traced to a mismatched .NET version; he also keeps a GitHub "snippets" repo of commands he forgets, and calls his weekly interesting-links post his own bookmarks. He says he shares not for credit but almost for selfish reasons. Giard says the same of his blog: notes to his future self, and his own perspective rather than a copy of the documentation.
• Put shared material in public, low-friction places like GitHub. After collecting job-search resources in a private Slack channel during his own search 2–3 years earlier, he moved them to a public GitHub readme that takes pull requests; his Ohio tech events repo has had pull requests from five or six people. He prefers this to LinkedIn, which not everyone uses or will log into, and says it is free of paywalls, walled gardens and algorithms. Giard adds that his own Midwest events list needed a modified blog engine to avoid a post date, where a repo would have been simpler.
• Sharing need not mean blogs or conference stages. Eaton suggests showing two or three teammates something, a lunch and learn, or a five-minute lightning talk at a meetup, and says a CodeMash lightning talk changed his own perspective; one person getting something out of it is enough. He urges anyone helped by a talk to tell the speaker.
• Start in the medium you are comfortable with. He says forced writing shows, so writers should write and speakers should start small; he prefers blogging because a post lasts and can be sent as a link, while a talk is "an hour and poof, it's gone". Everything he writes starts on paper, including talks scripted word for word before the slides. Of online critics, he stays off X, finds Mastodon engagement higher quality, and ignores the "well, actually" crowd.

2024-01-26 · open the original ↗

• This post includes paid content that is not available publicly. See the original source.

2024-01-25 · open the original ↗

• The fault is prescriptive architecture, not Clean Architecture itself. Miller, who founded Jasper FX Software the year before, objects to Clean or Onion Architecture templates used as infallible rules: four or five projects from the start, and a noun-based chain (invoice controller, invoice service, invoice repository, DTOs) in separate projects that scatters closely related code. His advice to technical leaders is to hand over a decision tree, not black-and-white rules, explaining why and when the advice does not apply.
• Broad horizontal layers block incremental modernization. Miller says swapping a library or database (NHibernate to the latest EF Core, or SQL Server to Cosmos DB, DynamoDB or his own Marten) means changing every handler in a layer at once, too big to sell to product owners. He wants loose coupling between modules so one can be upgraded at a time; Giard adds his own experience of a new column having to be carried through every layer, which Miller knows as the "wormhole anti-pattern".
• The full template is severe overkill on small services. Miller has repeatedly seen teams build a microservice of two or three controllers and about 12 endpoints with separate controller, service and repository projects, a mediator and base classes everywhere. In his view that erased the microservice's advantage: small enough to test purely with integration tests and so written as much simpler code.
• Prioritize test automation, now easier with disposable databases. Miller says layering is worth it only where it serves testability, and good test coverage keeps the architecture reversible so layers can be added later. Mocks arose because databases were hard to set up; engines that spin up in Docker and wipe state between tests let him choose integration testing over clean-architecture abstractions, and he calls mocks "overused", not evil.
• Organize vertically around use cases, then horizontally within each slice. Miller favours "vertical slice architecture" (Jimmy Bogard's term), grouping code by commands or events so one feature lives in one place. He credits Bogard's MediatR with breaking up bloated MVC controllers of 15–20 HTTP methods into single-message handlers, and offers his own Wolverine, a messaging and mediator tool with a 10–12-year lineage under earlier names, whose HTTP endpoint layer can replace MVC Core plus MediatR.

2024-01-24 · open the original ↗

• A workflow's triggers are whatever is listed under its on section. Giard extends the simple echo workflow from the previous episode, which had only workflow_dispatch (run manually by pushing a button). He points to the "events that trigger workflows" page in the GitHub Actions docs at docs.github.com, which lists the available events alphabetically with sample YAML to copy.
• Issue events can be narrowed to specific activity types. He copies the docs' issues sample, keeps only the opened and edited types and drops the milestone ones; the docs list others such as pinned, unpinned and closed.
• A push trigger can be filtered to a branch, folders or wildcards. He adds a push trigger limited to the main branch, noting the docs cover further filtering by folder and wildcard, and warns that indentation matters in the YAML. The result is three triggers, with the manual one kept so the workflow can still be run by hand.
• Both new triggers fired in his test. A commit made in the GitHub web editor to main showed up in the Actions tab under its commit message, queued while a VM container spun up and then ran; creating a new issue likewise started a run identified by the issue's name. Giard says most of the short run time was spent spinning up the container.

2024-01-24 · open the original ↗

• A GitHub Action runs jobs, made of steps, in response to triggers. Giard's examples of triggers are checking in code, creating an issue and clicking a button. The repository's Actions tab offers starter templates grouped by deployment, security, continuous integration and automation, made by GitHub, companies or the community. He starts from GitHub's "Simple workflow".
• In YAML, indentation defines the structure and errors break it. He covers the basics: # starts a comment, settings are name-value pairs, and each level (jobs, then the job name, then steps) is indented two spaces further than its parent. Bad indentation produces syntax errors. He keeps only the workflow_dispatch trigger, which adds a manual "Run workflow" button, and leaves runs-on: ubuntu-latest, noting you would change it for Windows or a specific Linux version.
• His demo workflow is two jobs that just echo text. Job one has a one-line script step and a multi-line script step; job two has one step. The file is committed under the .github/workflows folder, and he renames it from blank.yml to gcast-simple.yml. A manual run queues while an Ubuntu container spins up, and the run view shows each job, its steps and their output, which is also where errors would appear for troubleshooting.
• needs makes one job wait for another instead of running alongside it. By default the two jobs ran at the same time. Adding needs: job1 to job two makes them run one after the other, which Giard says suits a case like one job creating a file the next one uses. The rerun took about five seconds longer, and the run graph shows the dependency. He edited the YAML straight into main in the browser and says this is "probably not great for application lifecycle" but fine for a demo.

2024-01-24 · open the original ↗

• The host critiques another channel's install, not one Crosstalk did. The install, shown on Ghost Town Living, brought internet 900 ft down a mine shaft at Cerro Gordo, California. A NanoBeam AC took a feed from an on-property tower into a MikroTik hEX PoE lite, then a pair of Veracity Highwire ethernet-over-coax adapters (up to 200 Mbps, almost $450 together) took it down the shaft. The host offered in December 2020 to wire the town for free in exchange for filming, and repeats the offer.
• The coax adapters were unnecessary once power was run down the shaft. Since mains power already reached the bottom, the host argues a PoE switch could sit down there rather than pushing PoE over coax. A MoCA adapter would carry the network without PoE for about a third of the price and at higher speed, he says.
• He would have used two UniFi switches and armored fiber. With a US-8-150W at each end, armored outdoor fiber could go down the shaft, giving full gigabit, less exposure to lightning than copper, and one management interface for a very remote site. The switches handle both 24 V passive PoE and 802.3af/at. He estimates it at roughly $100–200 more, or more with spares.
• The first attempts failed on a spliced coax and an access point. After the coax was spliced, neither network nor power got through, so 1,000 ft of coax was replaced. A UAP-AC-Mesh-Pro then would not broadcast even on a PoE injector. A non-Pro AC Mesh taken from the hoist house worked at about 50 Mbps. The host would have put the non-Pro unit at the bottom from the start, since wireless mesh cannot pass through rock anyway.
• He would add resilience and reach beyond what was built. A preconfigured spare switch should stay on site, because the hoist house is dusty and very hot or cold, and a replacement could take days or weeks. He suggests Starlink (100–300 Mbps) as a backup WAN. A switch at the 900 level would also let cable run 200 ft up to the 700 level for the water-pump camera, which the build as shown cannot support.

2024-01-24 · open the original ↗

• CSS custom properties remove the need for a preprocessor for variables. Riviere says developers who learned CSS 10 or 20 years ago still reach for Sass or Less to get variables, but a property defined with a -- prefix and read with var() can hold fonts, colours or sizes; he defines a shared font fallback stack and colours named "paper" and "ink", which he prefers to foreground and background as clearer. Redefining a property on an element (headers, or one element's style attribute) overrides it for everything inside through the cascade, and browser dev tools show each variable's current value and where it came from.
• CSS Grid with named template areas is his favourite way to do page layout. He says Grid originated at Microsoft in Internet Explorer and reached every browser in about a month in 2018; he applies display: grid to a wrapper rather than the body, because third-party scripts inject elements into the body, names areas (side, content, footer) with grid-template-areas, assigns elements to them, and spaces them with gap, which applies only between rows and columns. Wrapping the grid in a min-width: 768px media query makes the single-column mobile view the default and the fallback.
• The :has() selector styles a parent by what it contains, without JavaScript. Riviere's example is highlighting the box around a label and an invalid input, which previously meant listening to the input and toggling a class on the parent in script; he demonstrates bordering any article that contains a tag link with a given data attribute. He says it now works in all major browsers, with Firefox the last to ship it, about a month before recording.
• Logical properties keep spacing correct in right-to-left languages. He shows that padding-left stays on the left when text is translated into Hebrew, which used to require direction-specific selectors; properties such as padding-inline-start, padding-inline-end, padding-block-start and padding-block-end attach spacing to where text starts and ends instead.

2024-01-22 · open the original ↗

• Microsoft's military team runs a tech boot camp for transitioning veterans. Gorman describes the Microsoft Software and Systems Academy, a 17-week program with three paths: cloud application development (ending with AZ-204), cloud administration (AZ-104), and a new cybersecurity path, which he says suits veterans who already hold clearances. Active-duty members take it under orders in roughly their last six months of service, honorably discharged veterans can also join (one of his students left three years ago), and there are Pacific and Central European regions, though one overseas student is attending his Central-time cohort from 7 p.m. to 3 a.m.
• Admission is selective and takes a couple of months. As Gorman understands it, applicants complete LinkedIn Learning courses, may once have had to pass a Pearson aptitude exam, and go through several interviews. He tells interested veterans to follow the Microsoft military team on LinkedIn for informational webinars and application deadlines; Giard gives the site as military.microsoft.com/mssa.
• The developer curriculum takes beginners from C# to Azure certifications. Gorman's current class is the 13th run of the cloud application track: 15 students, all virtual, four days a week, most of whom had never programmed. They start with C# console apps built around their own interests (one student's collectible Starbucks cups, used to teach inheritance), then Microsoft Learn's C# path with the free 80-question freeCodeCamp C# certification, data structures, and AZ-900 and AZ-204.
• Placement support includes a guaranteed Microsoft interview. Fridays, and one full mid-course week, go to professional development with Microsoft mentors on resumes and interviewing. Gorman says 8 of his first cohort's 15 got Microsoft offers and several students were placed at Southwest Airlines, a Microsoft partner, while his second cohort took longer to place in a weaker job market; he has students build GitHub portfolios and team MVC projects to stand out as juniors.
• Gorman calls this the most rewarding work he has done. He teaches as a subcontractor through Opsgility, his former employer and a Microsoft partner that manages cohorts, and cites seeing veterans move from military pay to developer salaries. He jokes, and says students confirm, that their hardest civilian adjustment is that nobody shows up to meetings on time.

2024-01-22 · open the original ↗

• This post includes paid content that is not available publicly. See the original source.

2024-01-20 · open the original ↗

• The pursuit of immersion led them from live-action film to animated VR. The Mojicas say their feature The Life Peddler shot every scene in one take to feel like watching real lives, and VR promised to put the viewer inside the story. With 360-degree 3D live-action cameras the viewer stays fixed in one spot, so to let viewers move around they had to build digital worlds with digital characters.
• Viewers choose which of several simultaneous storylines to follow. Their VR movies run four or five storylines at once; the demo has about 10 characters, each with a full story in roughly four to five minutes, meant as short, rewatchable episodes. Jose says the hard part is telling a story when the viewer can abandon the main character for a side story.
• A pre-recorded "reference video" lets two people perform every character. Because one long take allows no cuts, they first record themselves reading all parts and edit that into a synced, captioned reference video, a method they say took about a year to work out. They then perform each part separately while watching it, recording body motion apart from face and voice; characters built in iClone and Character Creator are driven by motion-capture suits, and odd bodies need care, since the bacon-wrapped avocado is all head and Jose must pivot his whole body to play it.
• They chose Unreal Engine for its easy path to a VR headset. Laurel says its VR template built and ran on a Meta Quest far more easily than Unity, and its blueprint system needs no low-level programming. The engine supplies collisions, gravity and interaction: viewers can pick up a pizza box or carry a lantern, and climb onto the roof or into trees; an artist builds assets in Blender on top of purchased graveyard and forest sets.
• The self-funded demo is aimed at future investors. It is a graveyard scene in a town of monsters who make bad low-budget horror films, with the Mojicas as the directors, and they hoped to finish it by March, speaking on 15 January 2024. The team is five people (the two of them, an artist, a gaming-degree programmer who writes the blueprints, and a social-media hire), with clips at baconwrappedavocado.com and on YouTube, and T-shirt sales planned.

2024-01-20 · open the original ↗

• Kubernetes won by building the largest coalition, not by being first. Burns says he, Craig McLuckie and Joe Beda began in fall 2013, went public at DockerCon in 2014 and hit 1.0 a year later, but it was 2016 or 2017 before they stopped arguing for it against Mesos, Docker Swarm and tens of other orchestrators. The mature ones were internal to large internet companies and not open source, Mesos was built for big data, and Docker focused on the single machine, while they felt they "had the puzzle box"; what made Kubernetes dominant, he says, was a safe place where any company could see a way to succeed.
• A new project starts with a demo people can actually use. Burns says the key early task is getting others to believe the vision, and a hands-on demo, not a PowerPoint, does that; his analogy is that people fixated on snowy passes and speed traps never leave Seattle, but show them a Walla Walla winery and they forget the six-hour drive.
• Open source was not optional if Kubernetes was to win. At Google he had to argue against making it a product that ran only on Google's platform, and a colleague accused him of getting the company to fund his startup; his answer was that the world is multi-cloud and hybrid and wants one solution, so a project that does not go everywhere is abandoned. He calls himself no zealot, saying "it wasn't this should, it was this has to be", and adds that open technology widens hiring, since someone with years on AWS can run workloads on AKS.
• The CNCF exists to provide vendor-neutral trust. Burns credits McLuckie with building the Cloud Native Computing Foundation under the Linux Foundation, launched a year after Kubernetes' release with Kubernetes as its flagship, so companies building on it need not fear trademark or IP claims; he cites Istio as the most egregious holdout, donated only about a year before recording. To avoid kingmaking amid a venture-funded "Cambrian explosion" of projects, and after pressure from VCs and connections began burning out governance people, CNCF settled on tiers: an unendorsed sandbox, a carefully judged incubating stage, and a handful of graduated projects.
• Kubernetes had to invent its governance as it grew. He says few open-source projects had been this successful, and neither Node.js nor the Linux kernel (which leaves much to distributors and grew up on mailed patches) fit a GitHub-native project that also did its own releases, testing and documentation; it needed elections and voting rules that large companies could not capture, a code of conduct with eviction, and an embargoed security-disclosure process. At Microsoft, his team now runs the Azure Kubernetes Service, packages AKS for edge and IoT, and keeps a dedicated team contributing upstream to Kubernetes and CNCF projects.

2024-01-17 · open the original ↗

• Gen 3 is excellent where Starlink is the only broadband option. In populated areas the host points to cable or fiber, and argues 5G home internet is faster and cheaper as a backup. He paid $599 for the dish on the Mobile roaming plan and will pause it and keep it in his RV for areas with no LTE. Unpausing bills the whole month. His Gen 1 plan went from $99 unlimited to $120 with a 1 TB cap, and he pays about $30 a month for T-Mobile Home Internet at about 400 Mbps.
• The dish is smaller overall, tougher, and has a detachable cable. It is slightly bigger than Gen 2 but has a manual kickstand instead of a motorized stand, so it takes up less space. It is rated IP67, against IP54 before. It ships with a 50 ft cable that can be swapped for an optional 150 ft one, where Gen 1's cable was fixed.
• Power draw is the host's biggest objection, especially off-grid. Starlink states an average of 75–100 W, against a 75 W maximum for Gen 2. He measured about 10 W for the router alone, a peak of 108 W while booting, and 35–48 W at idle. That is a real drain on RV batteries or solar, compared with about 10 W for his T-Mobile router.
• The router's two built-in Ethernet ports are the big improvement. Gen 2 needed a separate, often out-of-stock Ethernet dongle. The router is also 4x4 MU-MIMO Wi-Fi 6, which the host says mainly helps local transfers because Starlink speeds are about 100–200 Mbps. It can mesh up to three routers, and he believes it works with Gen 2 routers too. It has bypass mode, custom DNS and content filtering, which blocked adult sites when he tested it.
• Setup took about half an hour, and speeds were strong. Plugging it in took about 5 minutes, and connecting and updating took another 20–30. The host praises the app's real-time alignment guide, which corrected a 12° misalignment. He measured 155/9.3 Mbps in the Starlink app and 308/31.2 Mbps on speedtest.net, and 150–300 Mbps down across the house.

2024-01-15 · open the original ↗

• This post includes paid content that is not available publicly. See the original source.