Hi, my name is

Mark L Potter

I'm a Senior Linux Sysadmin with a focus on high performance computing.

I make lots of computers act like one big computer, wear kilts, dabble in Python for personal projects, and use Linux for everything.

About Me

I’m an HPC specialist with more than fifteen years experience in the supercomputing/HPC space and over twenty years experience in Linux system administration in corporate environments. Over these years I have held an RHCSA, done kernel customization, uncountable PXE deployments, and many complex software installations. I tend to automate anything I have to do more than twice. Due to the needs of my current gig, I have had to branch out and learn a good bit of M365 administration but I wouldn’t call myself an expert by any means.

Here are some of the things I am familiar with

Experience

Sr. Sysadmin - Seitel Inc
November 2021 - Present

I currently work as Sr. Sysadmin for Seitel Inc. My responsibilities include a small, ~100 node, HPC cluster and its infrastructure, a couple of TrueNAS file servers that handle ~1.5 PiB raw storage, a Proxmox VE hyperconverged environment with GPU passthrough, remote access for our entire workforce, the odd bit of networking, and monitoring for all of those bits and everything else.

  • Architected and implemented TrueNAS storage solution
  • Planned and executed move from in-house datacenter to colocation
  • Implemented Parallels RAS for remote access to Windows desktops and applications
  • Architected and implemented a Proxmox solution for remote 3D Linux workstations
  • Responsible for directly supporting all users at all levels of support
  • Standardized enterprise-issued devices to make support easier in a work from home environment
  • Implemented Netbox for IPAM and networking visualization
  • Implemented CheckMK for monitoring various infrastructure bits
Sr. Linux Administrator - Downunder Geosolutions
June 2019 - October 2021

At Downunder Geosolutions (DUG) I was a core part of the team that managed a complex infrastructure spanning four geo-disparate sites as well as over 8000 physical nodes local to Houston. The biggest challenge was implementing current knowledge at a much larger scale without overloading the infrastructure. All of the Linux folks were responsible for everything in the environment whether that was the diskless boot images, monitoring, or infrastructure projects like Proxmox VE.

  • Implemented a global FreeIPA solution to replace an aging OpenLDAP implementation
  • Architected a hyperconverged Proxmox VE solution for business critical virtual machines and containers
  • Developed proof of concept using Wazuh for security monitoring
  • Responsible for level 3 escalations of HPC cluster issues
  • Implemented a complete DNS solution for ad/malware/phishing blocking along with serving over 100,000 internal DNS entries using Ansible for deployment and Git for management
  • Member of the team responsible for maintaining the diskless boot image for over 12,000 compute nodes using an internally developed PXE deployment system built on bittorrent to allow easy scaling
Sr. HPC Architect - PCPC Direct
February 2014 – June 2019

PCPC Direct was a VAR that serviced the oil and gas industry as the majority of its business. My responsibilities varied from project to project based on current needs. It was a fast paced environment and required constant learning. Whether it was an RFQ that required learning and understanding a new technology or maintaining certifications the learning was constant. This is where I earned my RHCSA as well as many smaller Redhat certifications. A lot of the work involved travel and I truly enjoyed being on the road, meeting new people, and seeing new datacenters.

  • Technical lead for the development of a 3D accelerated, remote workstation and collaboration solution utilizing Red Hat Virtualization, Red Hat Cloud Suite, and Nvidia vGPU technology
  • Team Lead for teams managing multiple remote clusters in a managed services environment including all systems administration tasks, maintaining and tuning schedulers/resource managers, performance tuning, and system monitoring. Responsible for deploying multiple clusters at client sites ranging from single rack clusters to clusters utilizing hundreds of nodes
  • Responsible for onsite testing of non-managed clusters prior to hand-off to clients including local and remote sites
  • Developed an automated system using xCAT to deploy clusters, collect inventory, run burn in, and validate results in order to drastically reduce the man hours necessary to complete the integration and testing process
  • Architected a burn in suite utilizing FOSS tools to simulate various HPC workloads in order to reduce the number of failures after cluster delivery
  • Responsible for software stack architecture for all HPC related RFPs including OS, scheduler/resource manager, development tools, monitoring, and applications for all hardware vendors
  • Developed cluster administration documentation for clients as well as in-house use
UNIX Sysadmin - MD Anderson Cancer Center
April 2012 – October 2013

MD Anderson is likely the best cancer research hospital in the United States if not the world. While the work was roughly the same as other HPC jobs, the work was directly supporting cancer research. It was in this position that I gained familiarity with R and its intricacies. Unlike the oil and gas industry where everything is standardized, each researcher had their own preferences involving applications and languages. This was a fun challenge as multiple versions of Python, R, and Perl had to be maintained along with some of the standard genetics packages like Top Hat and Bowtie. With a stable cluster most of the team’s time was spent either installing new tools or updating current installations. The central apps repository was the largest I have worked with.

  • Responsible for maintaining a 336 node/8064 core HPC cluster using HP CMU
  • Development of cluster node images
  • Responsible for maintaining centralized installs for Perl, Python, R, and various NGS processing packages
  • Project lead for converting the cluster from CentOS 5.5 to RHEL 5.5 and from RHEL 5.5 to RHEL 6.2
  • Designed and deployed a cluster health monitoring system using Icinga and a cluster metric gathering system using Ganglia
  • Participated in the development and execution of a move of all computing resources from an older datacenter to a newer facility
  • Team lead for migration of the research and development environment to a new filesystem

Hobbies

It may be a little strange to list the operating system that I’ve based my career on as a hobby but the simple fact is that I still love Linux. I enjoy tinkering, distro hopping, and pretty much everything about being a Linux geek. I have multiple servers at home and all of them run Linux. I even do all of my gaming on Linux thanks to Steam’s Proton and Bottles. I turned a hobby into a career and haven’t looked back or abandoned the hobby. It is one of the joys I have in life and I can’t imagine it changing. I am always willing to help someone set up a Linux box whether it be to continue using older hardware or something like a purpose built server. I am a very lucky person, I turned something I loved doing into a career and I still love doing it!

Being huge fans of board games, my wife and I run a board game night at our church. This helps us build community and creates a place where we can socialize with people who enjoy the same things that we do. Board games run the gamut from being extremely competitive to requiring a great deal of cooperation. I prefer co-op games but am generally willing to play anything. We own a large variety of games and game night lets us experience games that we may have never seen. It is one of my favorite nights of the month. These are my current favorite games.

I started playing tabletop roleplaying games (TTRPG) with Advanced Dungeons and Dragons 2nd Edition when I was 17 years old. Over the last 30+ years I have developed lasting relationships and a fair number of skills. For quite some years I have mostly been a Game Master (GM). This has allowed me to learn how to bring together groups of disparate people to work towards a common goal as well as mediate conflicts between individuals. Staying current on the underlying systems requires a fair amount of continuous learning which helps keep me sharp. I have run various TTRPGs for groups of friends and even as team building at DUG. At present all of my games are online using the Foundry Virtual Tabletops. I keep current on the following TTRPG systems so that I can adequately run or play them.

Get in Touch

I am quite happy in my current position and am not looking for work but if you have a question or just want to drop me a note then go ahead and click to email me!