Newest thoughts →

March 2023

Two in-development tools/Python libraries that I have been working on are now out there. These are the start of a planned suite of Crossref API tools for interacting with our services.

To start, I've been building some utility libraries that we needed. The first is "longsight".

The second project is called "claws": the Crossref Labs AWS toolset. The boto3 library takes ages to import and can get messy, with different coders instantiating clients in different ways. This library centralizes these objects, giving just one entry point.

Longsight is a library that allows you to instrument functions with a decorator that gives you logging objects. These objects give structured JSON, can push to CloudWatch, extract correlation IDs in FastAPI contexts, and more. It basically handles best practice logging stuff.

Longsight is here: https://gitlab.com/crossref/labs/longsight

The two projects interact well with one another. Longsight will reuse (or even create) your claws objects, for instance. Together, these projects give us the baseline of observability that contemporary cloud applications need. They're not finished, but we've made a start!

Claws is here: https://gitlab.com/crossref/labs/claws

We're off to pick up Luka on Saturday! Poor chap has had such a tough life. I hope we can give him a nice home now where he doesn't have to worry.

"We'll protect the most vulnerable"

Except, I've just heard:

* CMDUs closing at end of June
* Covid anti-virals will now have a prescription charge
* Anti-virals taskforce is being disbanded
* Enhanced Protection Programme will not meet again

They're doing nothing.

@1goodtern @libdemdaisy

And that's a wrap! A few final checks tomorrow AM and this is done and going off to the Press, after several years of work.

Because digital text has executable contexts (code), LLMs that can publish (say web pages) are dangerous because they will be able to perform speech acts with real consequences. Virus/exploit writing will basically have a much lower barrier... and the model itself can use them.

Today has been some coding, a peer review, and then editing the final chapter of my next book... So nearly there now!

Doing a peer review this morning and next time I am requested to do this I will first ask: "do you use Editorial Manager for your MS management?"

If so, I'm not going to touch it with a barge pole. It's SO frustrating. I just want to give my review, not tell it my life history!

Great careers advice from The Sun. I've often thought that the stress-free life of an army officer was for me. 🤔

Congratulations to @erikketzan whose Thomas Pynchon and the Digital Humanities is now out in paperback with some ace endorsements

Today, I:

* refactored some code to use enums for clarity
* wrote some Terraform tests in terratest
* accidentally nuked my deployment using terratest
* reworked the Labs API proxy to use FastAPI on Lambda for route handling
* reworked all my unit tests to account for this

I also wrote a report on application observability via AWS CloudWatch in transient Lambda contexts where a Prometheus endpoint for scraping is not viable.

Three years ago, today, marked the start of a covid lockdown for most people in the UK. But we'd already been shielding for 2.5 weeks by that point; I missed my Leverhulme Prize award. And we're still in near complete isolation now as I remain at huge risk, with no end in sight.

Evusheld 2.0 (SUPERNOVA) brings some hope. But it's a way off. And the UK government probably won't buy it before the virus mutates again, if their past form is anything to go by.

I've been working today on the observability of the Labs API that I'm building.

Not gonna lie: Grafana dashboards make me feel like some supervillain monitoring everything from the secret cave.

Tomorrow, we will deliver a letter to the PM at No 10 Downing Street to mark the 3rd Anniversary of the first national lockdown in 2020, demanding better protection for the immunocompromised.

@MarkOak04974342 will be representing us, @evusheld4theuk.

I have not been able to be as involved in the campaign recently as I would like, owing to enormous health challenges, but this still remains of vital importance to me.

Further, MPs from all sides of the house will attend, including Tim Loughton MP, Daisy Cooper MP, and Andrew Gwynne MP.

The BBC should also be there to cover it.

We have also invited representatives from LUPUS UK, Kidney Research UK, and Action for Pulmonary Fibrosis, along with Tracey Allen representing the APPG for Vulnerable Groups to Pandemic.

Another week and another drop in kidney function. "It's deteriorated very quickly, hasn't it?" the dialysis unit said.

Also, grateful for clinicians phoning, but every time I get a call from a private number (inevitably the hospital), my heart sinks in the face of more bad news.

Arranged to speak with the dog shelter today... But they didn't call. 😔 Suspect they were just busy, but we're getting excited! Come on!

65 pages left to edit on my current book project. To be honest, this has been a total slog through illness to get it done, but having the end in sight feels great.

The knee-jerk response I've seen from quite a few people to ChatGPT is "ha! It's not even that good. This is no threat" etc.

I'm afraid to say that it will become good. It will become accurate. It will learn academic referencing. And probably quite soon.

At my wit's end negotiating transfer to local renal unit. Despair of the systems.

London hospital referred me. GP also referred me, using "ERS".

Local renal unit has no record of either.

Imagine being Jeremy Hunt and believing that the pressing problem that the budget needs to solve is ensuring that those who have earned enough to hit the lifetime pension allowance can accrue more. 🤦

A health update as I haven't posted one in a while. Things do not look very good, to be blunt. My kidneys are now sailing into the total failure zone, I have not cleared the BK virus that is destroying them, and I'm being referred to the local dialysis unit.

I still also feel very unwell a lot of the time at present, as the rheumatoid arthritis is not well controlled. I'm keeping relatively cheerful about it and life, but it's a bit of a blow given how hard we tried to beat the virus, with so much suffering involved.

But anyway: today I wrote the beginnings of what I hope will form the Crossref Labs API environment, where we will prototype new metadata fields and changes to the schema. It's an AWS Lambda function that proxies to the live API allowing us to intercept and modify the response.

Shown here is the preservation data being injected into an API result

It's all being developed in the open, here: https://gitlab.com/crossref/labs/lambda-api-proxy

My new role is amusing for my home life. I try to explain to my other half what I've done on any given day at work, but it's just word soup to her when I say "I optimized my AWS Lambda function to minimize cold start times".

Today's book in the post is Sarah Clegg's WOMAN'S LORE: 4,000 YEARS OF SIRENS, SERPENTS AND SUCCUBI, which sounds most interesting.

Today, I am going to start planning a staging environment for Crossref's scholarly metadata schema. At the moment, when we want to test/change new metadata aspects, we have no way of prototyping it. We want, now, to build a Lab version of the API where we can model new changes.

*spoiler potential for Endeavour*

The final episode of Endeavour should have been called The Thursday Murder Club.

ITV seems to have renamed its "ITV Hub" digital offering to "ITVX", which I initially assumed was some grubby adult spin off channel.

Well, today didn't really go as well as I had hoped. The rheumatoid arthritis flared badly, the horrific nausea/vomiting from pain returned, and I never got to electro-funk music. Oh well. Painkillers and try again tomorrow.

Netgear sells a router that costs £1,600. It hits a lot of my "wow I want that" buttons (but also my: "I can't afford that" button!) But what's most irritating is that it has a 10gbps WAN port... But only 2.5gbps or 1gbps LAN. So no device can use the full net speed. 🤷🏻

Really heartening to read of new @VersusArthritis research projects. This research is what makes my existence possible. Great charity. Please do support them if you can.

The political right win out either way by suppressing views in the BBC fiasco. Not only do they get to control the state media while it exists, but widespread anger at the corporation and its decline/collapse would suit many tories just fine.

Aaaargh.

Every time I think I have Airflow tamed, it throws new problems at me.

Turns out you can't use multiprocessing in an Airflow task. So the "local" mode in my Dask handler doesn't work... When run in Airflow.

500 deaths per year due to UK ambulance delays is bad and newsworthy - it's indicative of health service underfunding.

But 379 people died last *week* (and every week) of Covid and that's just the new normal, apparently.

@ernestopriego first made this point to me

OK, uplifting week on the health front didn't last.

Kidney results are much worse again. BK virus continues to attack them. Pretty dire and deteriorating quicker than expected.

Was hoping to make it past my birthday before I needed dialysis, but looking less and less likely.

However, this week has been uplifting for a very different reason: I think the abatacept is working! The blinding pain I have had for 6 months is abating. I feel like a different person.

Today has been an incredibly frustrating day on the technical front. I'm really close to having our infrastructure pipeline working - but bootstrapping git onto AWS managed Airflow is a total nightmare - especially when wrapped in GitPython. I've got one idea left to sort it!

Extremely frustrating that family of and carers for the most vulnerable will not be offered a booster jab this spring - only the vulnerable person themself. It just gets harder and harder for this group to stay safe.

Today I worked on the Crossref Labs infrastructure. I'm trying to build a pipeline using Airflow to drive a Dask cluster that autopulls source from GitHub and that can switch between local and distributed modes. Last hurdle is the garbage collector eating some objects...

I hate the phrase "cost of living crisis". It reifies it into something that apparently has "just happened" rather than it being the outcome of political choices. If the majority of people can't afford basic bills that's "poverty".

Very good news on the mature students front in the UK today as the equivalent learning qualification (ELQ) is set to be abolished (although not for several years). This is one of the core causes of Birkbeck's challenges as it blocked people getting loans to study with us.

Of course, I'd still rather that HE was NOT funded by personal loans in this country.

iNews really shocking me here.

‘I paid off my £440,000 mortgage in four years – here’s how I did it’

Answer: "I earned a lot and didn't have any outgoings".

Quite remarkable. My smartwatch measures "stress" using heart rate variability. Ever since I've been off the rheumatoid meds, my days have looked like the first image. This is last Friday. Now, I think, the new drugs are kicking in and last night is looking so much better!

Most likely all caused by pain...

An interesting call with immunology this week. They want to run genetic tests on me to ascertain whether my immune system problems could be permanently cured using a bone marrow transplant. It's very much an outside chance but would be life altering. The tests take up to a year.

Today is the third anniversary of our shielding from Covid. Apart from unavoidable hospital visits, I remain in pretty strict isolation, going into a fourth year. When the weather improves, we'll see friends outside.

I miss the old world - the one where I went to the pub with friends or a restaurant for dinner or had people round just to dance in our sitting room. I miss going to the theatre and music concerts. But I can't see how these will ever be safe for immunocompromised people again.

I can't afford to get Covid again, with Evusheld efficacy waning. My bout in January wasn't as bad as it could have been. But that's not saying much. I still ended up in hospital for 3 weeks! They said I'm prone to immediate reinfection because I don't develop an immune response.

Anyway, I am resigned to this limited life. But you can help to make the world safer for people in my condition. Wear a mask. Assume that the person next to you on the train is a cancer patient or a kidney transplant recipient. Don't kill them by going out when you have symptoms.

Does size matter? Here's a measure of size of Crossref member (number of deposits), counting the number in each size that has insufficient preservation. (i.e. members who have deposited between 101 and 500 DOIs do worst.)

(Just testing on a small subset of members at the moment)

Today is more data visualization work. I have to say, the streamlit python library is so good for this. Takes so much of the work out of drawing beautiful charts in an interactive web context.

Beginning the next big processing job for my digital preservation project. Iterating over all Crossref members, appraising preservation status across all archives for a sample of up to 1,000 DOIs each.

Next steps: wrangle this into a format conducive to nice visualizations...

This week's Endeavour had a great Laurel and Hardy joke, where Morse says he was "following in Hardy's footsteps in the West Country" to which Strange replies "well, that's another fine mess, eh?" Amusing.

After my tweet about Andrew Weatherall dying and then the apocalypse starting the other week, a colleague kindly sent me an AW tote bag!