Have Electricity, Will Travel
First, the really awesome takeaway from the Isaac Arthur video about portable power sources: biogamma-voltaics. An organism such as a fungus that is able to absorb gamma radiation and convert it into biochemical energy. That seems strange but it's difficult to artificially convert gamma radiation into electrical energy similiar to how solar cells convert visible light into electrical power. We already have examples of simple celled organisms surviving in high levels of ionizing radiation such as Chernobyl.
The most common portable power sources today are processes such as chemical batteries, mechanical flywheels and gas generators.
Using metamaterials it should be possible to create better atomic batteries that generate electrical power from radioactive decay. The video gives examples such as tritium betavoltaics and "diamond batteries" based on carbon-14. Small modular nuclear reactors can also be designed to generate power through fission.
Some of the more exotic sources of power which are theoretically possible include extracting energy directly from the vacuum of empty space. Perfect mirrors can "bottle" light. Concentrate enough light into a small enough volume and a Kugelblitz micro-black hole can be created and the Hawking radiation can be tapped for power.