Here is the main thing I've been trying to wrap my head around the past weeks.
ImE, almost all technically minded people view computation as substrate independent. That means it's algorithms, abstract computational steps, that count; it doesn't matter how they are implemented. The entire field of computer science is built on the premise of abstracting "Computation" or "Bits". The Turing Machine is like the ultra foundational model of computation and appears to be fully general, many other models have been proven equally powerful, some independently. And a TM is the quintessential abstract, substrate-independent model; you just write little symbols onto a tape, it doesn't matter whether it's a physical tape or transistors or a computer simulation.
As a corollary, everyone views the task of understanding the brain as the task of figuring out the right algorithms. Once you find the algorithms, it doesn't matter that they run on a digital computer rather than a brain. Computation is substrate independent.
(( And this is relevant for me because it's the primary argument against panpyschism. Take a human brain. Scan it. Upload it. The substrate is totally different. But if you scanned it with sufficient detail, the uploaded brain should carry out the same computational steps. That means if you scanned the brain of a philosopher who writes papers about consciousness, the uploaded brain should still write papers about consciousness. Surely, this means it must still have consciousness. Therefore, consciousness must be located at the algorithmic level rather than the physical level. Bits, not atoms. ))
But...
Heating up a metal and letting it cool slowly will change its internal structure. Viewed on that level of abstraction, this does computational work. You've taken some kind of a grid or lattice or whatever and rearranged it. Could be formulated as a purely abstract computational task. But this won't work if you use a different material. Suddenly, the substrate matters.
Similarly, take a quantum computer and start changing the substrate. See what happens. (This is why I want to understand exactly how they work.)
This will probably be the single biggest pill to swallow for my audience as well. But computation as substrate independent is simply not always true.
Now maybe substrate dependence is rare, so it's not a big deal. (though I'd argue that for the philosophical question, it's a massive deal even if it's rare.) Maybe nothing like that happens in the brain, so the normal approach is fine. Or maybe stuff like that happens in the brain ALL THE TIME as QRI thinks, and organic intelligence is all about using substrate-dependent computation. In this case, there is a massive, fundamental difference between biological brains and digital computers. This doesn't mean there's anything computers can't do, but they do it *differently*, and all of the people who are trying to reverse-engineer the neocortex have something coming for them. Or rather, they have nothing coming for them; their approaches will never work out.
Also digital computers may outright never have unified consciousness. here is something I would not have expected myself to say a year ago. But if consciousness requires substrate-dependent computation, that's the logical result.