H3-metal – Native MiniMax-H3 inference for Apple Silicon

(github.com)

246 points | by swyx 7 hours ago

11 comments

  • Meleagris 5 hours ago
    I've been using MiniMax H3 on my M5 Pro 64GB MacBook Pro through ComfyUI. It works extremely well.

    I had to modify the default ComfyUI workflows to use a GGUF quant (city96's ComfyUI-GGUF custom node, UnetLoaderGGUF in place of the stock loader) [0].

    I use the model labeled Q5_K_M. There is Q8_0 available as well, which is 34GB and fits fine in 64GB unified memory if you keep resolution modest.

    The main issue is speed, a ~9-second 480x864 clip at 20 steps takes me a bit over an hour. So this will be cool to try for the speed up alone.

    There's a lot of great information and workflows available to follow on the r/StableDiffusion subreddit.

    [0] https://huggingface.co/Abiray/MiniMax-H3-GGUF/tree/main/unet

    • antirez 59 minutes ago
      This implementation is much faster on my M5 Max, like a few minutes for the same video, but on an M5 Max with 128GB, didn't test on M5 Pro. About memory, could be executed on 64GB with a few changes.
      • Manfrednotfunny 49 minutes ago
        Memory bandwidtih between pro and max is double. 300gb/s vs. 600gb/s btw.
        • antirez 34 minutes ago
          Does not matter much in this case. GPU bound.
    • alexgoodhart 3 hours ago
      I wonder how much faster your m5 pro is compared to my M1 Max @ 64gb
      • yieldcrv 1 hour ago
        wait till the M7 bro you’re almost there, rumor has it that Apple is skipping the M6 but it still might be 2028
    • jonplackett 1 hour ago
      What is the quality of the output like compared to something like Veo?
    • Myzura 41 minutes ago
      How much free space do you have left after running this llm model? Have you tried to develop your own model with the M5?
  • myshapeprotocol 7 minutes ago
    Native inference optimization for Apple Silicon is such a game-changer for local-first workflows. Incredible performance work.
  • antirez 2 hours ago
    In the AMA Minimax said that H3 could support sparse attention, that would be a huge speedup! I wonder if there are any news on that. H3 is very cool. EDIT: testing a --sparse-attention optional mode based on what they said in the Reddit post.
  • linzhangrun 2 hours ago
    On my 128GB M4 Max Mac Studio, generating a 15s 480p video with MiniMax H3 in ComfyUI takes an hour and a half.

    Put Codex to work on deploying it now, hoping the speed can improve quite a lot :-) Thanks anyway

    • sscarduzio 2 hours ago
      Please keep up posted about the results!
      • linzhangrun 7 minutes ago
        First batch of quick test results: approximately 1/5 speed improvement
  • diddid 4 hours ago
    This is where the DGX spark makes up a bit of the ground it loses on llm work, diffusion and cuda go together like peanut butter and jelly.
  • TechSquidTV 6 hours ago
    This still requires 128Gb of memory, right? Me and my lowly 96Gb, like a commoner; missing out on the fun.
    • thehamkercat 6 hours ago
      From README:

      > On the 128 GB M5 Max, clean end-to-end image+audio and embedded-video+audio renders completed in 74.58 and 76.99 seconds respectively, each with about a 40.1 GB peak physical footprint and zero swaps.

      Looks like it uses 40GB? So your 96GB mac setup should work fine i guess (Model itself is 33B)

      • vunderba 5 hours ago
        This repo looks neat, but I hope they add some more clear benchmarks because that time (74.58s) is pretty meaningless given that the it/s (and total time) is highly dependent on mode (T2V vs I2V vs REF2V), resolution (0.4, 0.6mp, etc), duration (5-15 seconds), etc.
  • c0rruptbytes 5 hours ago
    wow antirez does not sleep
    • punnerud 3 hours ago
      Wow, had to check some of his other repos; his the one behind dump1090
    • mlmonkey 4 hours ago
      Understatement of the year :-D
    • behnamoh 4 hours ago
      when you have enough money to not have to worry about anything, you can go back to your hobbies. in this case, his hobby is programming.
      • azinman2 3 hours ago
        Being a world class talent is independent of financial situation.
        • afavour 1 hour ago
          Of course it isn’t. If you can’t afford to eat you can’t achieve any potential you might have. Financial stability is a gamechanger for everyone.
          • ionelaipatioaei 9 minutes ago
            That's stupid, if you're truly talented you'll solve the financial stuff in order to pursue whatever you want to do - if you don't then that's on you.
        • moshun 1 hour ago
          “I am, somehow, less interested in the weight and convolutions of Einstein's brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops."— Stephen Jay Gould
        • behnamoh 3 hours ago
          talent w/o financial stability is a battery w/o circuit.
      • stressback 4 hours ago
        "enough money not to worry about anything" haha
  • abhinai 6 hours ago
    How similar are Jeff Dean and Salvatore Sanfilippo?
    • onionisafruit 6 hours ago
      My favorite Jeff Dean fact is that he’s also antirez. Which reminds me of my favorite Salvatore Sanfilippo fact. He’s also Jeff Dean
      • hnfong 48 minutes ago
        The claims seem fitting from a user named "onionisafruit"...
      • tomcam 4 hours ago
        I'm totally following this
      • ralusek 4 hours ago
        Is that what the identity function is?
        • silisili 3 hours ago
          It's why javascript had to add the triple equals check...
          • muragekibicho 2 hours ago
            2 is not enough. 3 verifies the Dean-Sanfilippo correspondence.
    • robotresearcher 2 hours ago
      People are really good at stuff.

      I noticed on a bar TV the other day that some of the Chromecast screensaver landscape photo credits were to Peter Norvig. They were really lovely pictures.

      • SwellJoe 1 hour ago
        I've run into Peter Norvig twice. Once at a YC event; the other when I parked my motorhome in front of his house in Palo Alto for a couple of days while visiting a friend who happened to live on the same street (not on purpose, I didn't know it was his house, it was just where I found sufficient open street parking for a huge motorhome, big houses with fewer cars on the street than on my friend's block). I ran into him while walking my dog, he asked about the motorhome and we talked travel. He was lovely both times. Not everyone is nice about a big motorhome parking on their block, especially in California, but he was friendly.
  • yieldcrv 1 hour ago
    Alright I’ve been afraid to ask but have been having trouble finding

    What are some adult entertainment workflows in comfyui, I need best loras, best prompts to start with

    and the communities, are they on telegram or something?

    • bahmboo 29 minutes ago
      This is a healthy question. We want to use these tools for regular old human needs and desires.
    • Havoc 35 minutes ago
      There will Reddit subs for it though couldn’t tell you which off top of my head

      I’d personally steer clear of messaging platforms for this - who knows what one might stumble into there

  • tipiirai 4 hours ago
    I'd love to know what the alternatives are and how this is better
  • songhonglei1985 4 hours ago
    [dead]