GNU social JP
  • FAQ
  • Login
GNU social JPは日本のGNU socialサーバーです。
Usage/ToS/admin/test/Pleroma FE
  • Public

    • Public
    • Network
    • Groups
    • Featured
    • Popular
    • People

Notices by Fred Hebert (mononcqc@hachyderm.io)

  1. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Sunday, 19-May-2024 04:35:26 JST Fred Hebert Fred Hebert
    in reply to
    • Terra Field
    • Dr. Cat Hicks

    @grimalkina you apparently follow me on here already ;)

    I think you may like to hear that I used the paper (shared to me by @RainofTerra) as an inspiration for a discussion session with our on-call engineers titled “Can Stress, Fear, and Anxiety be useful in Ops, and if so, when?”—just hoping to make room for the topic.

    We ended up having a few people commenting along the lines of “I’m so glad to hear this isn’t just a ‘me’ problem,” which is great.

    In conversation about a year ago from hachyderm.io permalink
  2. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Sunday, 19-May-2024 04:35:23 JST Fred Hebert Fred Hebert
    in reply to
    • Terra Field
    • Dr. Cat Hicks
    • Carol Lee

    @RainofTerra @CSLee @grimalkina For me there’s been a gradual but definite shift between a view of engineers as stoic ever-correct beings whose every fiber is derived from pure analysis, towards the systems- and resilience-oriented approaches of well-intentioned beings bound by local rationality who dynamically help each other keep things from tipping over the edge of chaos.

    Funnily enough, it feels much better to live within the latter lens.

    In conversation about a year ago from hachyderm.io permalink
  3. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Friday, 12-Apr-2024 21:10:51 JST Fred Hebert Fred Hebert

    If anyone in the #erlang community has been using the rich compiler error option in Rebar3 over the last year, I’m looking for any feedback or opposition before turning it on by default in the next release: https://github.com/erlang/rebar3/pull/2881

    In conversation about a year ago from hachyderm.io permalink

    Attachments


    1. https://media.hachyderm.io/media_attachments/files/112/258/161/109/849/834/original/a6fd8f7b935f4c29.png
    2. No result found on File_thumbnail lookup.
      Turn rich compiler errors on by default by ferd · Pull Request #2881 · erlang/rebar3
      They've been enabled almost a year ago since 3.22.0. I've started gathering feedback on the Erlang Forums about it, and have been using it the whole time. Once a long enough period has gone without...
  4. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Wednesday, 31-Jan-2024 10:12:04 JST Fred Hebert Fred Hebert

    me: there is no reason why I would write more software, each new line of code is a burden I seek to avoid.

    also me: why yes I will use the bespoke slideshow software I wrote for fun for my next presentation, I just need to make sure it still runs fine on the newer runtimes and OS versions and—

    In conversation Wednesday, 31-Jan-2024 10:12:04 JST from hachyderm.io permalink
  5. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Friday, 24-Nov-2023 07:40:17 JST Fred Hebert Fred Hebert

    From a Reuters article:

    > OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks.

    This is such an incredibly bad definition from OpenAI that hours after reading it I still can't believe they'd pick something so bad.

    prosperity gospel-ass definition that fully lines up with the brainwormy tech shit effective altruists seem to keep churning out though, so sure why not.

    In conversation Friday, 24-Nov-2023 07:40:17 JST from hachyderm.io permalink
  6. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Tuesday, 21-Nov-2023 00:32:47 JST Fred Hebert Fred Hebert

    we're almost at the step of "every fridge needs to be connected and have an android running in it" of LLMs and it fucking sucks every time this happens.

    In conversation Tuesday, 21-Nov-2023 00:32:47 JST from hachyderm.io permalink
  7. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Thursday, 26-Oct-2023 05:31:56 JST Fred Hebert Fred Hebert

    There’s this pattern of “a neighbour is mowing the lawn” and one of them will go “oh yeah I should do that” and then he starts a bit later, and a third goes “is everyone mowing today? Guess I should” and before you know it there’s a chain of non-stop mowing that just eats the whole god damn day.

    Autumn is in and it's finally ending. It's one of the good things about cold weather.

    In conversation Thursday, 26-Oct-2023 05:31:56 JST from hachyderm.io permalink
  8. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Sunday, 10-Sep-2023 18:09:51 JST Fred Hebert Fred Hebert

    the sort of thing I wish more books stated is what ends up being the sort of unspoken thing that a lot of #DistributedSystems veterans do, which is: re-frame your system a different way until you do not need the fancy-ass distsys theory to make it work and it just survives common issues by being the right shape.

    It is generally easier to alter your roadmap than it is to engineer a novel distributed system that works the way you need it to.

    In conversation Sunday, 10-Sep-2023 18:09:51 JST from hachyderm.io permalink

    Attachments

    1. No result found on File_thumbnail lookup.
      http://www.shape.It/
  9. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Sunday, 27-Aug-2023 13:22:43 JST Fred Hebert Fred Hebert

    maybe we should teach people to program the way they teach martial arts, like only in the most desperate situations when all else failed should you resort to programming

    In conversation Sunday, 27-Aug-2023 13:22:43 JST from hachyderm.io permalink
  10. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Wednesday, 03-May-2023 12:51:00 JST Fred Hebert Fred Hebert

    Trust me, I can do way more damage to other computers than my own with terraform.

    In conversation Wednesday, 03-May-2023 12:51:00 JST from hachyderm.io permalink

    Attachments


    1. https://media.hachyderm.io/media_attachments/files/110/302/538/869/900/172/original/4b6732ae5d22a9af.png
  11. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Saturday, 25-Mar-2023 10:11:53 JST Fred Hebert Fred Hebert
    in reply to
    • Lorin Hochstein

    @norootcause I always found the Going Solid paper to be a good view of it and the implications of that model pushed a bit further (notes at https://cohost.org/mononcqc/post/888958-paper-going-solid)

    In conversation Saturday, 25-Mar-2023 10:11:53 JST from hachyderm.io permalink

    Attachments

    1. Domain not in remote thumbnail source whitelist: staging.cohostcdn.org
      Paper: Going Solid
      from https://cohost.org/mononcqc
      I had a quick chat earlier this week with a coworker which made me revisit Richard Cook's Going Solid: a model of system dynamics and consequences for patient safety [https://qualitysafety.bmj.com/content/qhc/14/2/130.full.pdf], written with the collaboration of Jans Rasmussen. It's a short paper that leans on and covers two concepts: Rasmussen's Drift Model and its ability to represent Going Solid as a concept, and it does so by applying it to situations in healthcare. "Going Solid" is an expression originally coming from the nuclear industry, when a boiler would become completely filled with liquid (solid). At that point, its operating characteristics become very different, and things become both dangerous and hard to control. The expression became a sort of piece of jargon, and Cook here expands it to mean in a more general sense that a situation in a system where components were loosely coupled suddenly become tightly coupled such that events can quickly go out of control. The paper aims to contextualize this concept within Rasmussen's drift model. The drift model is descriptive, not prescriptive, and describes the possible operating space for a sociotechnical system, based on an envelope bounded by 3 boundaries: economic failure, unacceptable workloads, and acceptable performance: Basic diagram of the drift model by Rasmussen, showing a rounded triangle. The left side is the acceptable performance boundary, the top side is an economic boundary, and the bottom side is an acceptable workload boundary. A dotted line within the triangle, to the left, defines a marginal performance boundary which defines a margin of error. The operating point is somewhere within the triangle, pushed toward the left by pressures coming from economic or workload factors [https://i.imgur.com/KCgOumi.png] > The operating point location is influenced by gradients that drive operations away from the workload and economic failure boundaries and towards the unacceptable performance (accident) boundary. Because the environment is dynamic, the operating point moves continuously; stability occurs when the movements of the operating point are small and, over time, random. Changes in the gradients (for example, increased economic pressure) move the operating point. The risk of an accident falls as distance from the unacceptable performance boundary increases. In practice, the precise location of the boundary of unacceptable performance is uncertain. Only accidents provide unambiguous information about its position. Because only accidents represent unambiguous information about your operating point, organizations tend to create a marginal boundary, which is enforced by social norms, and has people reel and try to bring the operating towards the center. Those are essentially "near misses" where you haven't had an accident but you knew you came close and operated in ways that felt unsafe. There are what we call High-Reliability Organizations (HROs), traditionally being things like nuclear power plants or the airline industry, which are known to be high-risk systems that nevertheless operate with long track records of safety, and then Low-Reliability Organizations (LROs), which still operate high-risk systems but tend to not do so reliably: Same chart as the previous one, but 3 areas are highlighted: a small one near the marginal boundary labelled HROs, a large one overlapping the marginal boundary labelled LROs, and an average-sized one comfortably within all boundaries labelled as a low-risk system [https://i.imgur.com/fTtVzLR.png] The key characteristic of an HRO in this view is that they have some awareness and agreement of where they are, and generally keep operating within a tight area with few large variations. LROs however would operate at the same point on average, but their overall spread and variation is much larger and therefore tend to cross the unacceptable workload boundary more often. Cook adds: > The marginal boundary location is variable because it is controlled by sociotechnical processes. Publicized painful accidents usually lead to stepwise movement of the boundary inwards. Long periods without such events may result in marginal boundary creep outwards. The gradients encourage organizations to "test" the validity of the marginal boundary by deliberately moving the operating point beyond it. Because this "flirting with the margin" does not immediately produce accidents, it can lead to incremental adjustment of the marginal boundary outwards. A zoom on the area between the acceptable performance boundary and the marginal boundary, showing the operating point of the system slowly drifting closer to the actual boundary, slowly redefining what the marginal boundary is. [https://i.imgur.com/M7SBXCz.png] The paper then turns its eyes toward the concept of coupling. Loosely coupled systems are those where activities and conditions in one part of the system have limited effect on those elsewhere. There is often buffering existing between parts of it, margins of manoeuver within which resources or efforts can be called upon to adjust to variations. Tightly coupled systems instead have many critical dependencies, which end up propagating effects widely and rapidly, and in ways that are difficult to anticipate. This in turn makes them harder to analyze, troubleshoot, and to intervene into. Tight coupling however tends to come with higher efficiency—optimizing away the slack buffers and utilizing resources more fully—and therefore an economical advantage. Coupling can also be created to avoid some frequent low-impact events. Specifically, the act of "going solid" is what happens when a generally loosely-coupled system reaches a saturation point and suddenly becomes tightly-coupled. The authors use hospitals and ICUs as an example of this, which is nice because it's not just mechanical components in a large piece of technical machinery: > Although tight coupling occurs in some hospital settings, hospital operations are usually loosely coupled. Individual units are independently staffed and have some degree of local autonomy. This arrangement produces operational "slack" that can buffer consequences of high workload in one unit. If an ICU is full, for example, it is possible to keep new critically ill patients in the emergency room or recovery room. This buffering capacity is lost, however, when the facility goes solid and all units are filled. Then even minor events in one unit may be major determinants of operations in the others. > > [...] > > To cite an example: a surgical procedure was cancelled after induction of anesthesia because a scheduled transfer of another patient out of the ICU was made impossible by deterioration of that patient's condition. The anesthetic was started because it had become routine to begin surgery in anticipation of resources becoming available rather than waiting for them to be available. The patient would have required an ICU bed for recovery after the procedure and the practitioners elected to halt the operation when it became apparent that no ICU bed would be available. They mention that "going solid" is a condition that once is hit may last for many weeks. It creates new types of work for practitioners, involves management in new ways, and create opportunities for new types of failure. In the hospital situation, you end up having to reorient a ton of work around managing patient discharge, accounting resources, creating new communication channels, and new behaviors emerge such as hiding or hoarding resources ("make a patient take longer to be discharged so it lines up with a new scheduled one coming in so you keep their bed without giving it to another department"). In fact, this condition may turn out to be profitable for the hospital which now spreads its fixed costs around more patients, despite all these challenges. However, when we look back at Rasmussen's drift model, tight coupling means that effects of disruptions are now much larger: A similar zoom on the left side of the triangle, showing an HRO with a lot of tight variations comfortably on the safe side of the marginal boundary, then going solid and having very wide motions within the margin, potentially into accident territory, and then coming back to a more loosely coupled state where it again operates safely with far smaller variations [https://i.imgur.com/qmXLvsa.png] > Because "going solid" is likely to occur when the operating point is already near the marginal boundary, the transition to tight coupling may allow otherwise minor changes in the operating point to propel the system beyond the marginal boundary and produce an accident. This is especially likely if there has been substantial marginal boundary creep during a prolonged accident-free period. Generally, safety efforts are concentrated around either pushing back the economic and workload boundaries or creating a force that counteracts their influence on the operating point, efforts to move the marginal boundary inwards (tolerating fewer things), or by moving the performance boundaries outward. What this paper highlights then is that there is also a potential benefit to better characterizing and understanding what your operating point is and how it shifts around over time. To do that, however, you need to have a general agreement about where you are, what's acceptable or unacceptable in order to properly negotiate the marginal boundary. So in general, there is value in surfacing signals to know the factors influencing where the marginal boundary is, what the responses are when you are crossing it, the degree of agreement of where you're operating now, and whether this is even accurate. The authors conclude: > Finally, the willingness of organizations to tolerate going and remaining solid for long periods is likely to encourage flirting with the margin crossing and marginal creep. The ability to map the operating point of individual systems through time and study of the factors influencing the marginal boundary location are likely to be productive lines of inquiry.
  12. Embed this notice
    Fred Hebert (mononcqc@hachyderm.io)'s status on Friday, 24-Mar-2023 10:11:25 JST Fred Hebert Fred Hebert

    We are releasing the well-poisoning machine, because if we don't do it first, someone else with even worse morals might do it faster.

    By being the first people to poison the well, we can know exactly what kind of toxins will be in the water such that we understand the symptoms when we inevitably make the whole village sick from it.

    Of course this has prompted our competitors to fire their ethics people and also poison the well even faster, but such is the inevitable march of progress!

    In conversation Friday, 24-Mar-2023 10:11:25 JST from hachyderm.io permalink

    Attachments


User actions

    Fred Hebert

    Fred Hebert

    Staff SRE @ honeycomb.io, Tech Book Author, Erlang Ecosystem Foundation board, Resilience Engineering fan. SRE-not-sorry.

    Tags
    • (None)

    Following 0

      Followers 0

        Groups 0

          Statistics

          User ID
          109341
          Member since
          24 Mar 2023
          Notices
          12
          Daily average
          0

          Feeds

          • Atom
          • Help
          • About
          • FAQ
          • TOS
          • Privacy
          • Source
          • Version
          • Contact

          GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.

          Creative Commons Attribution 3.0 All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.