jonathan@researchcomputingteams.org
Hi!
I’ve had to have a few different difficult discussions around our project this week, and while they were exhausting it’s been great to clear the air. And as far as I can tell they’ve strengthened rather than weakened the working relationships.
I also sat through a meeting with peer who essentially ran a denial-of-service attack on the meeting through his need to talk at length about anything even tangentially related to the topics at hand. It (almost) killed any chance that the meeting would accomplish anything. I’ve been trying to push myself to talk less and ask questions more during conversations; some days are better than others, but that experience has encouraged me to redouble my efforts.
It’s been a long seven days for a number of us, for a lot of reasons. I hope the week has been good to you and your team.
And on to the link roundup.
Managers: What do you do when your teammate shares their grief? - Lara Hogan
Hogan, taking lessons from her mothers pastoral care when she was going up, shares some humane steps you can take if someone unexpectedly shares grief with you over a significant tragedy in their lives. These steps are human for both the team member and you. The steps are:
Starting a new manager relationship - Sally Lait
A lot of us in research computing become managers by being promoted within an organization, so we sometimes have the advantage (and disadvantage..) of an extended handover process between the previous manager and you.
Lait here describes her preferred method for taking over a team in such a situation:
Asynchronous Meetings: Everything You Need to Know - Fellow App
As we get more and more comfortable with distributed teams, there’s increasing interest in written asynchronous team communication. It has the advantage of retaining a record and allowing people to contribute on their own schedule. Some things are hard to do asynchronously - it’s hard to imagine an asynchronous one-on-one being successful - but some are quite easy like status updates.
We know (from, for instance, open source) that complex decisions can be made with exclusively asynchronous communication; it’s even possible to set up to set up asynchronous meetings after a fashion - circulating an agenda with a deadline for items, have everyone contribute notes and discussions with a deadline, and proceed with decisions however you usually handle decisions.
There are definitely going to be discussions - such as one I had today where I badly misunderstood a fundamental point and didn’t realize it - where the high-bandwidth back-and-forth of synchronous discussion unblock things much more quickly than emails or shared documents. But a lot of routine communications can be done in writing and asynchronously. Getting used to working more in that mode is going to be something of a superpower for recruiting more distant employees, and teams who master it will also have a huge advantage in international collaborations.
Remote Onboarding Changes the New Hire Experience - Shane Hastie, InfoQ
More and more of our hires are going not going to be working in the same space as us, and that makes onboarding - something a lot of research computing teams are kind of sloppy about - even more important to do well. Hastie brings together a couple recent articles on the topic which encourage:
And some specific directions:
Research Computing Infrastructure and Researcher Engagement: Notes from Neuroscience - Bose, Antoniades & Pellman, PEARC ‘20
This is a very interesting conference paper from PEARC 2020 describing a discipline-focused research computing group to support a (then)-newly formed institute of neuroscience labs in Columbia. The goal was to focus on supplementing existing services by providing local infrastructure and training, and to facilitate partnering across labs.
So the group had a very clear focus, which was a huge advantage, and in addition prioritized a focus on tasks and services, as well as automation and self-service with possible. Those focusses meant they could communicate very clearly with labs around needs, speaking about tasks and services rather than implementation details, and it meant they could provide “bursting” to commercial clouds, all with a very small team. Future directions include assisting with software development, software consulting, and improving scheduling, monitoring, and visualization of existing workflows.
It’s really great to see what can be accomplished with a small team that has a laser-focus, and I’d love to hear more about how they developed their initial service catalog and built support for self-service and automation.
Towards making formal methods normal: meeting developers where they are - Alastair Reid, Luke Church, Shaked Flur, Sarah de Haas, Maritza Johnson and Ben Laurie
As mentioned before I’m quite impressed with how tooling like formal methods are advancing and I’m optimistic about their use for research software especially for subtle kernels of methods
This is an overview of where one group sees formal methods going, and if the topic interests you it’s worth reading for that alone; but it’s also worth reading as a product strategic plan:
A set of Common Software Quality Assurance Baseline Criteria for Research Projects - Orviz, Lopez, Duma, David, Gomez, and Donvito
Coming out of the EOSC Synergy effort, an extensive checklist of criteria for “production strength” research code, to be e.g. deployed as a service to communities in the INDIGO Data Cloud. The criteria are broken down into categories:
In most areas the actual recommendations aren’t that opinionated - e.g. processes must exist and be documented but they aren’t generally specified. This makes sense for a field as broad as research computing.
I haven’t seen such a comprehensive list before; this is a good conversation starting point conversation for any mature research software product.
COVID-19 Airborne Transmission Tool Available - Cooperative Institute for Research in Environmental Sciences (CIRES) 6 Google Sheets functions that do more than math - Justin Pot, Zapier
Here’s a controversial take - in research computing we tend to want to use Jupyter Notebooks + binder or RShiny apps to provide interactive calculations, even in situations where we could work faster and communicate more effectively with Google Sheets.
CIRES’s COVID-19 airborne transmission tool in Google Sheets is a nice example - it’s modelling functionality that’s only expected to be relevant for a couple years, people can trivially copy it and use it on their own, and the sort of decision makers who can set policies for institutions or jurisdictions are probably very familiar with and trust spreadsheets.
And Google sheets are actually pretty good for pulling data in from other sources and working with, as Pot’s article points out with IMPORTHTML and IMPORTFEED.
Interview with Meinte Boersma on Domain-Specific Languages - Federico Tomassetti
Boersma has a book coming out on Domain Specific Languages (DSLs). DSLs are starting to be seen more commonly in research software, as a way to make it easier to write code for a restricted but still rich problem. You could think of FEniCS as a DSL for finite element equation solving, or for a more explicit example Julia’s DifferentialEquations package.
Tomassetti’s interview with Boersma covers when a DSL is and isn’t a good solution to a problem, and also on the process of writing a book.
Running HPC workloads with Red Hat OpenShift Using MPI and Lustre Filesystem - David Gray, RedHat OpenShift
There’s increasing interest in running research computing workloads in Kubernetes environments. This blog post gives a high-level walkthrough on running GROMACS with MPI in OpenShift in particular.
One area that used to be complex for HPC-type workflows in kubernetes used to be access to high-performance POSIX file systems. Operators - Kubernetes’ way of encapsulating complex operations elements of a workload - now exist for lustre and MPI jobs, making that easier.
What’s interesting is that the FS is actually the easy part; for decent network performance, you currently have to bust out of the cloud-native SDN for performance. It’s interesting to see the different reaction to that in different communities - in cloud-native land this is treated as being deeply suspect, as the software defined networking is an important security consideration, while the HPC world takes a much more YOLO approach to such considerations.
Relatedly, there’s a new (I think) best practices guide for running MPI jobs on Google Cloud.
3rd OpenMP Users Developer Conference - 1-2 Dec, Online, Free
Free workshop on OpenMP; full day hands on tutorial on the 1st in EU/UK friendly covering the basics and OpenMP for GPUs, followed by a talks in the UK afternoon on the 2nd, covering code generation, load balancing, ARM, and the OpenMP roadmap.
For those involved in doing or supporting computational fluid dynamics, Dyke’s 1982 “An Album of Fluid Motion” is now available for free online. This is a lovely and beautiful book for developing intuition about fluid flow, especially (but not exclusively) incompressible flows.
The pinnacle of human achievement in wordprocessing software, WordPerfect for DOS, has been updated. Reveal-codes forever.
Free preview of an Apress book on Data Parallel C++ covering C++ and SYCL.
A HOWTO on routinely backing up your emails from an IMAP server.
How to setup secure ssh via public URL through a firewall with nginx, lets encrypt, and ssh reverse tunnelling.
RCEs exist for git-lfs on windows - best get patched if your team or users are using that combination.
Syncious, an HPC-specific cloud provider.
Swift is an interesting language but it hasn’t yet gotten (or, really, earned) much traction in research computing. With really excellent numerics work going on, and now a pretty sophisticated looking roadmap for concurrency (not parallelism, to be clear, but concurrency) I wonder if that might slowly start changing.
