Originally I used Pandera, but it had several issues last
* Mypy dependency and really bad PEP compliance
* Sub-optimal runtime check decorators
* Subclasses pd.DataFrame, so using e.g. .assign(...) makes the type checker think it's still the same type, but now you just violated your own schema
So I wrote my own library that solves all these issues, but it's currently company-internal. I've been meaning to push for open-sourcing it, but just haven't had the time.
Interesting article, but about halfway through what was a convincing analysis of an ongoing diesel shortage turned into a fear piece on the US/China relations and a corresponding economic collapse.
Diesel production capacity is limited and shrinking. Demand is there, albeit sticky (due to certain demand sources that are not easily electronified) and shrinking at a slower pace (due to slowing global economic growth). Rather than discussing a hypothetical blockade of Chinese oil imports, I wish the author would have addressed more probable short term outcomes re: supply/demand shock triggers and possible market plays.
Feel like this was a missed opportunity to look at recent geopolitical events under a more nuanced lens.
Schema inference: yes, Arc infers the schema automatically from incoming data (both for MessagePack and Line Protocol). Each measurement becomes a table, and fields/tags map to columns.
Schema evolution: supported. New fields can appear at any time, they’re added to the Parquet schema automatically without migration or downtime.
Custom partitions: currently partitioning is time-based (hour-level by default), but custom partitioning by tag or host or whatever is planned. The idea is to allow you to group by any tag (e.g. device, region) in the storage path for large-scale IoT data.
Roadmap: absolutely. Grafana data source, Prometheus remote write, retention policies, gRPC streaming, and distributed query execution are all in the works.
We are going to start to blogging about it, so, stay tune.
Would love any feedback on what you’d prioritize or what would make adoption easier for your use case.
My use case isn't IOT, but about once a month I get a massive data dump from a vendor. Think tens of millions of rows and 100+ columns. Cleaning, ingesting and querying this data via standard RDBMS is a slow and brittle process. There is a time series aspect, but partitioning across other keys/groups is critical.
I don't think Spotify and streaming killed the music subculture, it's still very much alive but requires more intention to find.
Back in the day how did you find new music? Pre-2000s it was likely MTV/radio for mainstream, or word-of-mouth/local events for niche genres. Nowadays Spotify and streaming services have supplanted the former for mainstream music. Finding new music outside the recommended engines requires a little more effort in knowing where to look. There are a lot of Internet radio programs (shout out to The Lot and Rinse.FM) and smaller record labels that do an amazing job at curating local and diverse sounds.
These days it's never been easier to start your own label or publish a track. Rock-'n'-roll is probably still alive (unfortunately I don't know that modern scene well), but assembling the necessary equipment and people to start a band is a big hurdle requiring practice, space and coordination. So I can see more wanna-be artists opting for pop/electronic having shorter turnarounds to a finished product.
>> I don't think Spotify and streaming killed the music subculture, it's still very much alive but requires more intention to find
No, I think music as a subculture is dead.
When I was a kid in the 80's I would sneak out of the house and go to a hardcore punk show that was put on by kids for kids with no adult involvement whatsoever.
Right now, on a Saturday night, where is there an all-ages music show going on anywhere in the US where the kids in the pit are 14, 15 years old and no adult knows or cares that they're there? There's a culture now, but there's no subculture. Most kids are watched too closely for that to happen now, which is good! Mostly. And subculture doesn't require a credit card and a subscription plan.
How did I find new music? Word of mouth was good, cassette tapes that your friends made you. Going to shows. A little bit from Night Flight. The Decline of Western Civilization. Urgh! A Music War.
Compilation records. American Youth Report, Flipside Vinyl Fanzine, the Mystic record comps, Rat Music For Rat People, the Blasting Concept, Dope, Guns, and Fucking in the Streets, Let Them Eat Jellybeans, maximumrocknroll. Forced Exposure. No New York. Going to record stores "in the city".
"Right now, on a Saturday night, where is there an all-ages music show going on anywhere in the US where the kids in the pit are 14, 15 years old and no adult knows or cares that they're there?"
I used to go to those kinds of shows in my late teens. For what it's worth, I mostly am out of that because a) I like calmer music and b) the kids should have their own space.
But that doesn't mean that I don't run into small all-ages punk shows. There's usually one going on in Durango, CO a couple weekends a month that I have seen. I'm certain that in larger places the same thing is happening even though I don't know about them.
Consider that older folks not knowing about the show is kind of the sine qua non of what you're looking for, so I am no surprised that I, an "older folk", don't know about them.
But I know enough to know they are there. It's easy to find new music if you're looking for it.
Maybe go read or review Hesse's "Journey to the East" if you want a longer version of what I am getting at.
Tell me where the music subculture is living and I'll move there right now. When we were kids it was Austin, Brooklyn, and Philadelphia for the insane IDGAF kids. For awhile it was Paris and maybe parts of Spain. Today I can't think of definitive hub for, "real musicians."
I think self-publishing is the problem. Making music on laptops is neat and everything but where the model in the 1990's was giving the bedroom rock hopeful group eg. Pixies, Nirvana, Smashing Pumpkins, Creation Records a million dollar record deal which gave a livelihood to the kids making the music the new model is, "have young artists self-finance their own careers and reward them with exposure when they produce something worthwhile with the hope that maybe their music gets licensed for a film." Touring isn't lucrative for many groups. Many tours are self-financed. Not often mentioned is that musician's a group notoriously deprived of healthcare due to healthcare being tied to traditional employment.
How could we combine the best parts of Johnny Marr's idea of, "being a working musician" while still affording young talented musicians the livelihoods and opportunities presented by the music industry of, "yesteryear?" My feeling always was in expanding the musician's reach into the world of pedagogy and, "play as a means of meaningful research." Delia Derbyshire comes to mind. Brian Eno half comes to mind. There's a better thing but it requires institutions and social democracy-- it requires a society with the social sensitivity to not envy or disdain, "people who make weird noises for a living and get to travel the world." The United States is not that right now unfortunately. The western world is in crisis and needs music but it lacks the scaffolding to create, "great musicians and bands."
As much as the world needs another John Lennon right now much more we need Brian Epsteins that can create John Lennons, Mick Jaggers, and Peter Noones with pen strokes. Where are the Don Kirshners of the world creating product groups like The Monkees and The Archies? I can tolerate greed if we can get another Smiths, Beatles, or another Paul Weller. I can tolerate another Andy Warhol is he'll produce another Lou Reed.
> Tell me where the music subculture is living and I'll move there right now. When we were kids it was Austin, Brooklyn, and Philadelphia for the insane IDGAF kids.
There are still geographical centers for certain genres of music. Austin is still a hub for psych/indie/alt music, New Orleans is where you want to be if bounce is your thing, etc.
But from my perspective, music subculture moved from TV and radio to the internet long ago. I no longer have "120 Minutes" telling me what its creators think is good, but I continue to hear great new music via TikTok and Instagram direct from artists and fans.
> Today I can't think of definitive hub for, "real musicians."
If you had to pick one physical location, that'd probably be Nashville (and not just for country). Other hubs would include L.A., NYC, London, Miami, Atlanta, Austin, and New Orleans.
Give me 5 noteworthy groups from Austin and I'll check them out. Mostly as an expert I don't see anything coming from the, "post Austin Psych Fest wake" that I should care about. Even as someone, "that played that shitty festival" with Joel Gion and all the rest (Black Angels, Loop) it wasn't Glastonbury in 95'; it wasn't anything. Find me a literate person that plays Vox 12-string and I'll form a band with you. I'll wait.
As for your second reply all those places are great but unaffordable for the young hopefuls. That's the filter that keeps the kids locked out. In the 70's moving to New York was, "Suicide" (Get it? If you don't I'd appreciate you'd just upvote this and stop reading. I'll provide a link below.) This is a filter that locks a lot of really talented kids out of the ecosystem.
I'd love to see a production grade release of this package with async and BCP support. MSSQL has always been a second class citizen (and rightfully so) amongst its open source peers and now it's nice to see MS dedicating resources to this project. They've got some catching up to do, but the alpha benchmarks look quite promising so far!
I think the Catholic church's actions could be rationalized through the lens of preserving influence. Descartes' ideas could be viewed as fostering a mindset of self-reliance and independence which is antithetical to a dependence on the church.
In other more heavily regulated industries, whistleblowers are fortunately compensated and protected for raising such ethical issues. I wonder how far tech can go before we start to see similar government agencies and rules put in place to do the same.
