The issue that lurks right over the horizon of possibility is whether increasing complexification in generatively encoded “intelligent machines” could instantiate some form of consciousness. I argue that the most probable answer is “yes”. The system would become auto-referential, and in this way, acquire a “sense of self”. Leaving aside more deeply philosophical discussions on the topic, at the most basic level this means that the system would develop an awareness of its internal state and of external conditions and be able to discriminate between itself and things that are not itself. This is an important step, as it would lead to relationality – a set of functions that provide resonance or dissonance with particular types of (internal and/or external) environmental conditions, reinforcement and reward for achieving certain goals or states, and in this way a sense of what neuroscientist Antonio Damasio has called “a feeling of what happens“; in other words, a form of consciousness (and self-consciousness).
There is much that we can do with neuroscience, its techniques and technological tools, but in each and every case, it is important to consider what should be done, given the attractiveness and limits of neuroscientific knowledge, socio-cultural realities, extant – and newly developing – moral constructs, and the potential to use any scientific and technological tool to evoke good or harm. At the fore is the need to regard neuroscience as a human endeavor. Therefore, we are responsible for the relative rightness and/or wrongness of the ways that neuroscientific knowledge and interventions are employed. Brain research and neurotechnology extend the boundaries of self-understanding, and may alter the way we view and treat both humans and non-human beings (e.g., animals, artificially intelligent machines, etc). Furthermore, neuroscience and neurotechnology provide means to control cognition, emotion, and behavior. While beneficent motives may drive the use of such capabilities, neuroscience is not enacted in a social vacuum, and thus, such interventions and manipulations are subject to the often-capricious influences of the market and political power. Accordingly, we must ask how these goods and resources will be employed, distributed, and what effect this will incur on individuals, groups, and society.