> The letters you already know, telling you what they sound like

So… letters?

If you think a cryptographically signed image can’t be JPEGed 5 times and shared on Facebook for misinformation then here’s a relevant XKCD for you:

https://imgs.xkcd.com/comics/security.png

No no the invisible hand of the market will fix it, you see. Car manufactures will become so big and pay so many taxes that we could pave every road with rail.

The cool thing about polars is that you can conditionally collect expressions over many layers of business logic, and then compute the result at the end. Doing this in SQL ends up in a hodgepodge of strings and trimmed ends to please the syntax. You can also pretty effortlessly write quite complex conditionals directly in polars, and bridge it easily to the surrounding python.

I find that SQL is only easier to read with minimal abstraction, but as soon as the project gets bigger SQL becomes an unwieldy island of different that has served its purpose after we’re done with reading/writing the data.

This sounds interesting! Do you have a specific example by any chance or blog post/doc references?

It’s just the lazy/expression part of the API, which is really the bread and butter of polars, rather than just being “replacement syntax” for pandas. This allows you to tap into abstraction that SQL can’t keep up with:

  import polars as pl

  # 1. Base Dataset
  lazy_df = pl.LazyFrame(
    {
      "store_id": ["S01", "S02", "S03", "S04", "S05"],
      "revenue": [5000.0, 2400.0, 15000.0, 900.0, 3200.0],
      "margin": [0.45, 0.30, 0.60, 0.15, 0.50],
      "tx_count": [120, 45, 300, 20, 85],
      "returns": [5, 12, 45, 2, 8],
    }
  )

  # 2. Define Layer Abstractions
  def get_kpi_layer() -> list[pl.Expr]:
    return [
      (pl.col("returns") / pl.col("tx_count")).alias("return_rate"),
      (pl.col("revenue") / pl.col("tx_count")).alias("avg_order_value"),
    ]

  def get_threshold_layer(thresholds: dict[str, list[float]]) -> list[pl.Expr]:
    return [
      (pl.col(col) > limit).alias(f"is_{col}above{int(limit)}")
      for col, limits in thresholds.items()
      for limit in limits
    ]

  def get_interaction_layer(numeric_cols: list[str]) -> list[pl.Expr]:
    return [
      (pl.col(a) / (pl.col(b) + 1e-5)).alias(f"ratio_{a}per{b}")
      for i, a in enumerate(numeric_cols)
      for b in numeric_cols[i + 1 :]
    ]

  def get_segmentation_layer() -> list[pl.Expr]:
    return [
      pl.when(pl.col("margin") > 0.4)
      .then(pl.literal("High"))
      .otherwise(pl.literal("Low"))
      .alias("margin_profile")
    ]

  # 3. Consolidate and Execute Single Graph Pass
  thresholds = {"revenue": [1000.0, 5000.0, 10000.0], "tx_count": [50, 100, 200]}
  numeric_cols = ["revenue", "margin", "tx_count", "returns"]

  expr_pool = [
    *get_kpi_layer(),
    *get_threshold_layer(thresholds),
    *get_interaction_layer(numeric_cols),
    *get_segmentation_layer(),
  ]

  final_df = lazy_df.with_columns(expr_pool).collect()

awesome, thanks!

Well, I prefer the polars version. And now if I want to reuse the CTEs elsewhere, I have to reach out to another tool like DBT or hand roll something to do string manipulation.

... does it? I don't think it does, even in this form.

And now write it such that all the conditions and transformations are injected into the string (somehow) rather than written in explicitly. Much worse.

Really? You've written out all the ratios and thresholds manually. If a user wanted to change the set of thresholds the polars way is far superior. In what way do you consider this better?

I don't know if my code is just "complex", but I find that Luna on max ignores the surrounding style and completely ignores logical consequences of a change, like just writing `del arg1, del arg2, ...` instead of dropping it from the surrounding code. All LLMs make questionable decisions at times, but Luna requires so much guidance that it's faster to just type it out yourself. What kind of routine tasks can one accomplish with such a model?

Do you have code formatters, linters and static analysis?

I can get extremely dumb models to get our code style correct because of those guard rails and a specific style document.

The US does not lead in broad markers of "social good" like life expectancy, happiness index or social mobility, so what exactly is this KPI useful for?

I've discovered so many great artists through Spotify, and been to many of their shows (which Spotify also helps surface with their "Live Events" feature). I regularly get artists with less than 100k listeners in my Discover Weekly, which I then look up, and find that they did a concert on KEXP, and the comments all proclaiming "I knew good music wasn't dead!".

So I can say for sure that I don't share the slop sentiment. Sure, there's some generic filler every now and then, but any non-cohesive mix, as opposed to a carefully curated album is going to have songs that naturally become "fillers".

Just now, I discovered that Spotify snuck in "The Juan Maclean - Running Back To You (2014)", which is a cover of "Brian Bennet - Solstice (1978)". Neither songs are completely unknown, but they're also not exactly Bruno Mars.

I don't know if this is pure luck or algorithmic rhythm analysis, but for me it's working.

If my manager's manager's manager decides, for any number of reasons, to buy a product which is using US-based technology, then I don't reasonably have the option to "vote with my feet", but I do have a fallback called Regulation, which works its way all the way up the chain automatically.

Why would you ever expect that you would have a choice of what products you use at work? You are being paid to work there. And yes, if you feel that strongly, you absolutely do have the option to vote with your feet at any time.

> Why would you ever expect that you would have a choice of what products you use at work?

Similar arguments were made to defend Harvey Weinstein.

> And yes, if you feel that strongly, you absolutely do have the option to vote with your feet at any time.

People shouldn’t have to switch jobs because they don’t want to provide Bill Gates with their phone number.

> Similar arguments were made to defend Harvey Weinstein.

We're now comparing a SaaS product versus people you work with? I don't think ChatGPT will ever physically assault you.

> People shouldn’t have to switch jobs because they don’t want to provide Bill Gates with their phone number.

This is why the EU has very strong employee protection laws; in most countries here there are government agencies that will help you, you're not at the mercy of the HR department.

>Why would you ever expect that you would have a choice of what products you use at work?

Umm Because of worker protection laws? For example the Betriebsrat has the right to have a say in this. Second what if the product is very much inferior and the whole company would be more productive with another product?

> You are being paid to work there.

Yes and that means if I see an improvement opportunity or if I have concerns I will voice them. I'm being paid to improve the company, not be a cog in the machine.

It’s their whole business:

“Stop studying Japanese. Start speaking it.”

> click here to buy our pro AI prompts

Ah, a tale almost as old as the one of the lake itself.

Off topic but the way the four “program…” words line up on an iPhone is truly mesmerizing.

a typographical river