
Site Reliability Engineer @Apple . SLOgician of sociotechnical systems. Opinions are my own.
London, England
Joined December 2010
- Tweets1.4K
- Following455
- Followers925
- Likes2.7K
Pavlos Ratis retweeted
Apple announces LLM in a flash: Efficient Large Language Model Inference with Limited Memory
paper page: huggingface.co/papers/2312.1…
Large language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks. However, their intensive computational and memory requirements present challenges, especially for devices with limited DRAM capacity. This paper tackles the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters on flash memory but bringing them on demand to DRAM. Our method involves constructing an inference cost model that harmonizes with the flash memory behavior, guiding us to optimize in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Within this flash memory-informed framework, we introduce two principal techniques. First, "windowing'" strategically reduces data transfer by reusing previously activated neurons, and second, "row-column bundling", tailored to the sequential data access strengths of flash memory, increases the size of data chunks read from flash memory. These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively. Our integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory.
Pavlos Ratis retweeted
If you're SRE or SRE-adjacent, it would be great to spend some time filling out catchpoint.com/sre-survey -- one of the few ways we can get cross-company readings on what's happening with the profession.
Thank you for your time in advance!
Pavlos Ratis retweeted
Are you an SRE looking to make a difference in your org? Use research, data, and math to slay MTTR. Find it in the 2022 VOID Report: thevoid.community/report
Pavlos Ratis retweeted
Would be curious to hear from practising SREs whether or not this captures their experiences
Are you a software engineering director in charge of some Site Reliability Engineers (SRE) and wondering what they’re doing - or _should_ do? Then @niallm has some guidelines for you...
stanza.systems/post/what-doe…
Pavlos Ratis retweeted
If you are responsible for production infrastructure this is for you: thevoid.community/report
Pavlos Ratis retweeted
I feel it is a great curated resource for SREs
Great work @dastergon 👏
teapawt.org/dastergon/status/15141…
🎉 The awesome-sre repo turns six today! I hope it’s helped folks and teams to get started with SRE. github.com/dastergon/awesome… #SRE #SiteReliabilityEngineering
Pavlos Ratis retweeted
Awesome Site Reliability Engineering by @dastergon
github.com/dastergon/awesome…
#devops #devsecops #kubernetes #cicd #k8s #linux #docker #sysadmin #automation #Azure #infrastructureascode #cloudcomputing #serverless #terraform #ansible #yaml #sre
Pavlos Ratis retweeted
The SRE Book did a lot of things. Were enough of them good?
@lauralifts has takes!
stanza.systems/post/should-w…
Pavlos Ratis retweeted
Just in case you’re looking, my org at Apple is hiring SREs, engineering managers, infrastructure managers, a system’s reliability tech lead, a cloud SRE manager, build engineers and much more. These aren’t full time remote roles, but im happy to provide more info in dms
Pavlos Ratis retweeted
I'm a big believer that great products are a result of healthy autonomy. It means less interdependencies between teams, strong API/data boundaries, mature capacity planning, the devops model where devs are operators of their services.
Pavlos Ratis retweeted
. @dastergon gave a talk at #SREcon21 on the "Lessons Learned Using the Operator Pattern to Build a Kubernetes Platform,"
usenix.org/conference/srecon…
youtube.com/watch?v=F-MLsAYb…
Pavlos Ratis retweeted
usenix.org/publications/logi…
USENIX's login magazine very kindly agreed to publish a slightly elaborated, text version of my keynote from last year's SRECon EMEA. Thanks to @lauralifts, @NYCDubliner and @systemician amongst many others for review + support.
Pavlos Ratis retweeted
# Writing for Engineers
Writing is a key skill to master if you want to grow your reach beyond the team level. Over the past year I have noted how many SWE struggle with this task, and helped a few to improve. Here are my top takes:
- Blog heinrichhartmann.com/posts/w…
- Thread 👇
Pavlos Ratis retweeted
Still a classic!
🎉 The awesome-sre repo turns six today! I hope it’s helped folks and teams to get started with SRE. github.com/dastergon/awesome… #SRE #SiteReliabilityEngineering
Pavlos Ratis retweeted
The curse of Cassandra was to be correct, but never believed; the curse of systems thinkers is to be correct, but never valued.
Part 1 of 2.
blog.relyabilit.ie/the-curse…
Pavlos Ratis retweeted
Lessons learned - Using the Operator pattern to build a #Kubernetes platform by @dastergon
#CNCF #SRE
youtu.be/F-MLsAYbF5I
Pavlos Ratis retweeted
As a senior engineer, you should learn how to be less reactionary and more strategic. Reactionary behavior creates tactical solutions. Tactical solutions create more problems. Learning to think big and fix foundational issues sometimes makes hard problems disappear.
Pavlos Ratis retweeted
What sociological factors are risks to software reliability?
I can imagine a few (burnout due to pager fatigue, toil or cognitive overload), but I'm curious what the SRE/DevOps community have experienced.
Please respond in thread and share/retweet
Pavlos Ratis retweeted
Hearing the word resilience, resiliency, and resilient thrown around a lot recently. See: resilience-engineering-assoc…




















