Which of these are photos vs. computer-generated images? (credit: Olivia Holmes et al./ACM Transactions on Applied Perception) (credit: Olivia Holmes et al./ACM Transactions on Applied Perception)
As computer-generated characters become increasingly photorealistic, people are finding it harder to distinguish between real and computer-generated, a Dartmouth College-led study has found.
This has introduced complex forensic and legal issues*, such as how to distinguish between computer-generated and photographic images of child pornography, says Hany Farid, a professor of computer science, pioneering researcher in digital forensics at Dartmouth, and senior author of a paper in the journal ACM Transactions on Applied Perception.
“This can be problematic when a photograph is introduced into a court of law and the jury has to assess its authenticity,” Farid says.
Training helps … for now
In their study, Farid’s team conducted perceptual experiments in which 60 high-quality computer-generated and photographic images of men’s and women’s faces were shown to 250 observers. Each observer was asked to classify each image as either computer generated or photographic. Observers correctly classified photographic images 92 percent of the time, but correctly classified computer-generated images only 60 percent of the time.
The top row images are all computer-generated, paired here with their photographic matches below (credit: Olivia Holmes et al./ACM Transactions on Applied Perception)
But in a follow-up experiment, when the researchers provided a second set of observers with some training before the experiment, their accuracy on classifying photographic images fell slightly to 85 percent but their accuracy on computer-generated images jumped to 76 percent.
With or without training, observers performed much worse than Farid’s team observed five years ago in a study when computer-generated imagery was not as photorealistic.
When humans can no longer judge what’s real
“We expect that human observers will be able to continue to perform this task for a few years to come, but eventually we will have to refine existing techniques and develop new computational methods that can detect fine-grained image details that may not be identifiable by the human visual system,” says Farid.
The study, which included Dartmouth student Olivia Holmes and Professor Martin Banks at the University of California, Berkeley, was supported by the National Science Foundation.
* Legal background
In 1996, Congress passed the Child Pornography Prevention Act (CPPA), which made illegal “any visual depiction including any photograph, film, video, picture or computer-generated image that is, or appears to be, of a minor engaging in sexually explicit conduct.”
In 2002, the U.S. Supreme Court ruled that the CPPA infringed on the First Amendment and classified computer-generated child pornography as protected speech. As a result, defense attorneys need only claim their client’s images of child pornography are computer generated.
In 2003, Congress passed the PROTECT Act, which classified computer generated child pornography as “obscene,” but this law didn’t eliminate the so-called “virtual defense” because juries are reluctant to send a defendant to prison for merely possessing computer-generated imagery when no real child was harmed.
Abstract of Assessing and Improving the Identification of Computer-Generated Portraits
Modern computer graphics are capable of generating highly photorealistic images. Although this can be considered a success for the computer graphics community, it has given rise to complex forensic and legal issues. A compelling example comes from the need to distinguish between computer-generated and photographic images as it pertains to the legality and prosecution of child pornography in the United States. We performed psychophysical experiments to determine the accuracy with which observers are capable of distinguishing computer-generated from photographic images. We find that observers have considerable difficulty performing this task—more difficulty than we observed 5 years ago when computer-generated imagery was not as photorealistic. We also find that observers are more likely to report that an image is photographic rather than computer generated, and that resolution has surprisingly little effect on performance. Finally, we find that a small amount of training greatly improves accuracy.
Vibrotactile actuators in prototype smart glasses (credit: Joseph Szczerba et al./Proceedings of the Human Factors and Ergonomics Society)
Human factors/ergonomics researchers at General Motors and an affiliate have performed a study using a new turn-by-turn automotive navigation system that uses haptic cues (vibrations) to the temples to communicate information to drivers on coming turns (which direction and when to turn), instead of distracting voice prompts or video displays.
They modified a prototype smart-glasses device with motors in two actuators (on the right and left side of the head) that buzz to indicate a right or left turn and how far it is, indicated by the number of buzzes (1 at 800 feet away, 2 at 400 feet, and 3 at 100 feet).
National Advanced Driving Simulator (NADS) MiniSim software (credit: NADS)
Using a driving simulator, each participant drove three city routes using a visual-only, visual-plus-voice, and visual-plus-haptic navigation system. For all three system modalities, the participants were also presented with graphical icons for turn-by-turn directions and distance.
Sample turn-by-turn direction icon (credit: Joseph Szczerba et al./Proceedings of the Human Factors and Ergonomics Society)
The researchers found that effort, mental workload, and overall workload were lowest with the prototype haptic system. Drivers didn’t have to listen for voice instructions or take their eyes off the road to look at a visual display. Drivers also preferred the haptic system because it didn’t distract from conversation or audio entertainment.
The results indicate that haptic smart-glasses paired with a simplified icon-based visual display may give drivers accurate directional assistance with less effort.
Mazda 2015 with GPS audio and video display (credit: Landmark MAZDA)
As noted in “Up to 27 seconds of inattention after talking to your car or smartphone,” two studies by University of Utah researchers for the AAA Foundation for Traffic Safety found that a driver traveling only 25 mph continues to be distracted for up to 27 seconds after disconnecting from highly distracting phone and car voice-command systems. The 27 seconds means a driver traveling 25 mph would cover the length of three football fields before regaining full attention.
According to the Multiple Resource Theory developed by Christopher D. Wickens in Theoretical Issues in Ergonomics Science (open access), multiple tasks (such as use of navigation systems while driving) performed via the same channel can result in excessive demand that may increase cognitive workload (and risks of an accident).
The new human factors/ergonomics haptics research was conducted by Joseph Szczerba and Roy Mathieu from General Motors Global R&D and Roger Hersberger from RLH Systems LLC. It was described in a paper in Proceedings of the Human Factors and Ergonomics Society September 2015.
Abstract of A Wearable Vibrotactile Display for Automotive Route Guidance: Evaluating Usability, Workload, Performance and Preference
Automotive navigation systems typically provide distance and directional information of an ensuing maneuver by means of visual indicators and audible instructions. These systems, however, use the same human perception channels that are required to perform the primary task of driving, and may consequently increase cognitive workload. A vibrotactile display was designed as an alternative to voice instruction and implemented in a consumer wearable device (smart-glasses). Using a driving simulator, the prototype system was compared to conventional navigation systems by assessing usability, workload, performance and preference. Results indicated that the use of haptic feedback in smart-glasses can improve secondary task performance over the conventional visual/auditory navigation system. Additionally, users preferred the haptic system over the other conventional systems. This study indicates that existing technologies found in consumer wearable devices may be leveraged to enhance the user-interface of vehicle navigation systems.
Elite X-Plane General Aviation Dream Package flight simulator system (credit: Xforce PC)
You can learn how to improve your novice pilot skills by having your brain zapped with recorded brain patterns of experienced pilots via transcranial direct current stimulation (tDCS), according to researchers at HRL Laboratories.
“We measured the brain activity patterns of six commercial and military pilots, and then transmitted these patterns into novice subjects as they learned to pilot an airplane in a realistic flight simulator,” says Matthew Phillips, PhD.
The study, published in an open-access paper in the February 2016 issue of the journal Frontiers in Human Neuroscience, found that novice pilots who received brain stimulation via electrode-embedded head caps improved their piloting abilities, with a 33 percent increase in skill consistency, compared to those who received sham stimulation. “We measured the average g-force of the plane during the simulated landing and compared it to control subjects who received a mock brain stimulation,” says Phillips.
“Pilot skill development requires a synthesis of multiple cognitive faculties, many of which are enhanced by tDCS and include dexterity, mental arithmetic, cognitive flexibility, visuo-spatial reasoning, and working memory,” the authors note.
Measuring brain activity to infer learning
Neuroimaging and tDCS experimental setup for stimulation. EEG locations are denoted in blue, fNIRS sources (red), and detectors (green). tDCS electrodes are denoted in purple (cathodes) and yellow (anodes). C and D: Predicted electric field intensities. (credit: Jaehoon Choe et al./Front. Hum. Neurosci.)
The study focused on a working-memory area — the right dorsolateral prefrontal cortex (DLPFC) — and the left motor cortex (M1), using continuous electroencephalography (EEG) to monitor midline frontal theta-band oscillatory brain activity and functional near infrared spectroscopy (fNIRS) to monitor blood oxygenation to infer neuronal activity.
The researchers used the XForce Dream Simulator package from X-Force PC and the X-plane 10 flight simulator software from Laminar Research for flight simulation training.
Previous research has demonstrated that tDCS can both help patients more quickly recover from a stroke and boost a healthy person’s creativity; HRL’s new study is one of the first to show that tDCS is effective in accelerating practical learning.
Phillips speculates that the potential to increase learning with brain stimulation may make this form of accelerated learning commonplace. “As we discover more about optimizing, personalizing, and adapting brain stimulation protocols, we’ll likely see these technologies become routine in training and classroom environments,” he says. “It’s possible that brain stimulation could be implemented for classes like drivers’ training, SAT prep, and language learning.”
HRL Laboratories, LLC | Enhanced Training Through Neurostimulation
Abstract of Transcranial Direct Current Stimulation Modulates Neuronal Activity and Learning in Pilot Training
Skill acquisition requires distributed learning both within (online) and across (offline) days to consolidate experiences into newly learned abilities. In particular, piloting an aircraft requires skills developed from extensive training and practice. Here, we tested the hypothesis that transcranial direct current stimulation (tDCS) can modulate neuronal function to improve skill learning and performance during flight simulator training of aircraft landing procedures. Thirty-two right-handed participants consented to participate in four consecutive daily sessions of flight simulation training and received sham or anodal high-definition-tDCS to the right dorsolateral prefrontal cortex (DLPFC) or left motor cortex (M1) in a randomized, double-blind experiment. Continuous electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) were collected during flight simulation, n-back working memory, and resting-state assessments. tDCS of the right DLPFC increased midline-frontal theta-band activity in flight and n-back working memory training, confirming tDCS-related modulation of brain processes involved in executive function. This modulation corresponded to a significantly different online and offline learning rates for working memory accuracy and decreased inter-subject behavioral variability in flight and n-back tasks in the DLPFC stimulation group. Additionally, tDCS of left M1 increased parietal alpha power during flight tasks and tDCS to the right DLPFC increased midline frontal theta-band power during n-back and flight tasks. These results demonstrate a modulation of group variance in skill acquisition through an increasing in learned skill consistency in cognitive and real-world tasks with tDCS. Further, tDCS performance improvements corresponded to changes in electrophysiological and blood-oxygenation activity of the DLPFC and motor cortices, providing a stronger link between modulated neuronal function and behavior.
The “stentrode,” created by the University of Melbourne’s Vascular Bionics Laboratory, adapts an off-the-shelf self-expanding stent to include a recording electrode array. The device is delivered to the brain through blood vessels in the neck, thus avoiding many of the risks associated with traditional placement of neural implants through open-brain surgery. (credit: University of Melbourne)
A DARPA-funded research team has created a novel minimally invasive brain-machine interface and recording device that can be implanted into the brain through blood vessels, reducing the need for invasive surgery and the risks associated with breaching the blood-brain barrier when treating patients for physical disabilities and neurological disorders.
(credit: University of Melbourne)
The new technology, developed by University of Melbourne medical researchers under DARPA’s Reliable Neural-Interface Technology (RE-NET) program, promises to give people with spinal cord injuries new hope to walk again.
The brain-machine interface consists of a stent-based electrode (stentrode), which is implanted within a blood vessel next to the brain, and records the type of neural activity that has been shown in pre-clinical trials to move limbs through an exoskeleton or to control bionic limbs.
The new device is the size of a small paperclip and will be implanted in the first in-human trial at The Royal Melbourne Hospital in 2017.
The research results, published Monday Feb. 8 in Nature Biotechnology, show the device is capable of recording high-quality signals emitted from the brain’s motor cortex without the need for open brain surgery.
“We have been able to create the world’s only minimally invasive device that is implanted into a blood vessel in the brain via a simple day procedure, avoiding the need for high risk open brain surgery,” said Thomas Oxley, principal author and neurologist at The Royal Melbourne Hospital and Research Fellow at The Florey Institute of Neurosciences and the University of Melbourne.
Stroke and spinal cord injuries are leading causes of disability, affecting 1 in 50 people. There are 20,000 Australians with spinal cord injuries, with the typical patient a 19-year old male, and about 150,000 Australians left severely disabled after stroke.
The University of Melbourne | Stentrode in action
Implantable stent with electrodes
Co-principal investigator and biomedical engineer at the University of Melbourne, Nicholas Opie, said the concept was similar to an implantable cardiac pacemaker — electrical interaction with tissue using sensors inserted into a vein, but inside the brain.
Stentrode with 8 × 750 micrometer electrode discs (yellow arrow) self-expanding during deployment from catheter (green arrow). Scale bar, 3 mm. (credit: Thomas J. Oxley et al./Nature Biotechnology)
“The electrode array self-expands to stick to the inside wall of a vein, enabling the researchers to record local brain activity. By extracting the recorded neural signals, we can use these as commands to control wheelchairs, exoskeletons, prosthetic limbs or computers. In our first-in-human trial, that we anticipate will begin within two years, we are hoping to achieve direct brain control of an exoskeleton for three people with paralysis,” he said.
Thought control
“Currently, exoskeletons are controlled by manual manipulation of a joystick to switch between the various elements of walking — stand, start, stop, turn. The stentrode will be the first device that enables direct thought control of these devices.”
Professor Clive May, neurophysiologist at The Florey, said the data from the pre-clinical study highlighted that the implantation of the device was safe for long-term use. “Our study also showed that it was safe and effective to implant the device via angiography, which is minimally invasive compared with the high risks associated with open-brain surgery.
The authors note that “avoiding direct contact with cortical neurons may mitigate brain trauma and chronic local inflammation,” subject to additional evaluation.
The University of Melbourne | Stentrode: Moving with the power of thought
In addition to DARPA, the research was supported by Australia’s National Health and Medical Research Council, the U.S. Office of Naval Research Global, The Australian Defence Health Foundation, The Brain Foundation, and The Royal Melbourne Hospital Neuroscience Foundation.
Lighter, more agile exoskeleton helps the paralyzed to walk
Steven Sanchez, who was paralyzed from the waist down after a BMX accident, wears SuitX’s light, more agile Phoenix exoskeleton. (credit: SuitX)
Meanwhile, in related research (also based on initial funding from DARPA), SuitX, a spinoff of UC Berkeley’s Robotics and Human Engineering Laboratory robotics lab, introduced last week the Phoenix — a new lighter, more agile and lower-cost manually controlled exoskeleton.
The Phoenix is lightweight and has two motors at the hips and electrically controlled tension settings that tighten when the wearer is standing and swing freely when they’re walking. Users can control the movement of each leg and walk up to 1.1 miles per hour by pushing buttons integrated into a pair of crutches. It’s powered for up to eight hours by a battery pack worn in a backpack.
Developed from the Berkeley Lower Extremity Exoskeleton (BLEEX), the Phoenix is one of the lightest and most accessible exoskeletons available, according to SuitX. It can be adjusted to fit varied weights, heights, and leg sizes and can be used for a range of mobility hindrances. At $40,000, it’s about the half the cost of other exoskeletons that help restore mobility.
Abstract of Minimally invasive endovascular stent-electrode array for high-fidelity, chronic recordings of cortical neural activity
High-fidelity intracranial electrode arrays for recording and stimulating brain activity have facilitated major advances in the treatment of neurological conditions over the past decade. Traditional arrays require direct implantation into the brain via open craniotomy, which can lead to inflammatory tissue responses, necessitating development of minimally invasive approaches that avoid brain trauma. Here we demonstrate the feasibility of chronically recording brain activity from within a vein using a passive stent-electrode recording array (stentrode). We achieved implantation into a superficial cortical vein overlying the motor cortex via catheter angiography and demonstrate neural recordings in freely moving sheep for up to 190 d. Spectral content and bandwidth of vascular electrocorticography were comparable to those of recordings from epidural surface arrays. Venous internal lumen patency was maintained for the duration of implantation. Stentrodes may have wide ranging applications as a neural interface for treatment of a range of neurological conditions.
An illustration of how a femtosecond laser pulse is delivered to the target point between an axon and a neuronal soma (cell body) (credit: the authors)
University of Alberta researchers have developed a method of connecting neurons using ultrashort laser pulses. The technique gives researchers complete control over the cell connection process and could lead to new research and treatment methods, including physical reattachment of severed neurons right after injury, the researchers say.
The team’s findings are published in the open-access Nature journal Scientific Reports.
Procedure
After putting two neurons in a special solution that prevents them from sticking together, the researchers brought them into contact with each other and delivered femtosecond (10-15 seconds) laser pulses to the meeting point of the two cells, causing them to establish solid bonds and form a common membrane at the targeted area.
(Left): An illustration of the phospholipid bilayers of the neuron soma and axon (the attachment region is designated with a circular spot — not the laser focal point). (Center): The laser pulse’s high intensity causes a reversible destabilization of both phospholipid layers. The generated free ions (red) and free electrons (orange) cross the center nonpolar region and break bonds between the fatty acid hydrophobic tails. (Right): The relaxation process results in the formation of new stable bonds and formation of singular, hemifused, cell membrane only at the targeted connection point. (credit: Nir Katchinskiy et al./ Scientific Reports)
The cells remained viable and the connection strong. It took the neurons just 15 milliseconds to stick to each other; the process would have taken hours to occur naturally.
“The preservation of the viability of the neural network will allow researchers to study new complex pathophysiological processes, such as neurogenesis, Wallerian degeneration, segmental demyelination, and axonal degeneration,” the authors note.
Abstract of Novel Method for Neuronal Nanosurgical Connection
Neuronal injury may cause an irreversible damage to cellular, organ and organism function. While preventing neural injury is ideal, it is not always possible. There are multiple etiologies for neuronal injury including trauma, infection, inflammation, immune mediated disorders, toxins and hereditary conditions. We describe a novel laser application, utilizing femtosecond laser pulses, in order to connect neuronal axon to neuronal soma. We were able to maintain cellular viability, and demonstrate that this technique is universal as it is applicable to multiple cell types and media.
(Left): Control rabbit brain, showing neuropil near the CA1 band in the hippocampus. (Right): Vitrified rabbit brain, same location. Synapses, vesicles, and microfilaments are clear. The myelinated axon shows excellent preservation. (credit: Robert L. McIntyre and Gregory M. Fahy/Cryobiology)
The Brain Preservation Foundation (BPF) has announced that a team at 21st Century Medicine led by Robert McIntyre, PhD., has won the Small Mammal Brain Preservation Prize, which carries an award of $26,735.
The Small Mammalian Brain Preservation Prize was awarded after the determination that the protocol developed by McIntyre, termed Aldehyde-Stabilized Cryopreservation, was able to preserve an entire rabbit brain with well-preserved ultrastructure, including cell membranes, synapses, and intracellular structures such as synaptic vesicles (full protocol here).
The judges for the prize were Kenneth Hayworth, PhD., Brain Preservation Foundation President and neuroscientist at the Howard Hughes Medical Institute; and Prof.Sebastian Seung, PhD., Princeton Neuroscience Institute and Computer Science Department.
First preservation of the connectome
“This is a milestone in the development of brain preservation techniques: it is the first time that the preservation of the connectome has been demonstrated in a whole brain (prior to this only small parts of brains have been preserved to this level of detail),” said the BPF announcement.
“Current models of the brain suggest that the connectome contains all the information necessary for personal identity (i.e., memory and personality). This technique is not the same as conventional cryonics (rapidly freezing the brain), which has never demonstrated preservation of the ultrastructure of the brain. Thus the winning of this prize represents a significant advance in the methods available for large scale studies of the connectome and could lead to procedures that preserve a complete human brain.
Kenneth Hayworth (KH) (President of the Brain Preservation Foundation (BPF)) and Michael Shermer (member of BPF advisory board) witnessed (on Sept. 25, 2015) the full Aldehyde Stabilized Cryopreservation surgical procedure performed on this rabbit at the laboratories of 21 Century Medicine under the direction of 21CM lead researcher Robert McIntyre. This included the live rabbit’s carotid arteries being perfused with glutaraldehyde and subsequent perfusion with cryoprotectant agent (CPA). KH witnessed this rabbit brain being put in -135 degrees C storage, removal from storage the following day (verifying that it had vitrified solid), and KH witnessed all subsequent tissue processing steps involved in the evaluation process. (credit: The Brain Preservation Foundation)
“The key breakthrough was the rapid perfusion of a deadly chemical fixative (glutaraldehyde) through the brain’s vascular system, instantly stopping metabolic decay and fixing all proteins in place by covalent crosslinks. This stabilized the tissue and vasculature so that cryoprotectant could be perfused at an optimal temperature and rate. The result was an intact rabbit brain filled with such a high concentration of cryoprotectants that it could be stored as a solid ‘vitrified’ block at a temperature of -135 degrees Celsius.”
Winning the award also required that the procedure be published in a peer reviewed scientific publication. McIntyre satisfied this requirement and published the protocol in an open-access paper in the Journal of Cryobiology.
3D microscope evaluation of the rabbit brain tissue preservation (credit: Brain Preservation Foundation)
The Brain Preservation Foundation plans to continue to promote the idea that brain preservation following legal death, by using scientifically validated techniques, is a reasonable choice for consenting individuals to make. Focus now shifts to the final Large Mammal phase of the contest, which requires an intact pig brain to be preserved with similar fidelity in a manner that could be directly adapted to terminal patients in a hospital setting.
The 21st Century Medicine team has recently submitted to the BPF such a preserved pig brain for official evaluation. Lead researcher Robert McIntyre has started Nectome to further develop this method.
“Of course, [the demonstrated brain preservation procedure] is only useful if you think all the relevant information is preserved in the fixation,” said Anders Sandberg, PhD., of the Future of Humanity Institute/Oxford Martin School. “Protein states and small molecule chemical information may be messed up.”
GPA | Will You Preserve Your Brain?
Background and significance (statement by BPF)
Proponents of cryonics have long sought a technique that could put terminal patients into longterm stasis, the goal being a form of medical time travel in which patients are stabilized against decay with the hope of being biologically revived and cured by future technologies. Despite decades of research, this goal of reversible cryopreservation remains far out of reach — too much damage occurs during the cryopreservation itself.
This has led a new generation of researchers to focus on a more achievable and demonstrable goal–preservation of brain structure only. Specifically preservation of the delicate pattern of synaptic connections (the “connectome”) which neuroscience contends encodes a person’s memory and identity. Instead of biological revival, these new researchers often envision a future “synthetic revival” comprising nanometer-scale scanning of the preserved brain to serve as the basis for mind uploading.
This shift in focus toward “synthetic” revival has completely transformed the cryonics debate, opening up new avenues of research and bringing it squarely within the purview of today’s scientific investigation. Hundreds of neuroscience papers have detailed how memory and personality are encoded structurally in synaptic connections, and recent advances in connectome imaging and brain simulation can be seen as a preview of the synthetic revival technologies to come.
Until now, the crucial unanswered questions were “How well does cryonics preserve the brain’s connectome?” and “Are there alternatives/modifications to cryonics that might preserve the connectome better and in a manner that could be demonstrated today?” The Brain Preservation Prize was put forward in 2010 to spur research that could definitively answer these questions. Now, five years later, these questions have been answered: Traditional cryonics procedures were not able to demonstrate (to the BPF’s satisfaction) preservation of the connectome, but the newly invented “Aldehyde-Stabilized Cryopreservation” technique was.
This result directly answers what has for decades been the main skeptical and scientific criticism against cryonics –that it does not provably preserve the delicate synaptic circuitry of the brain. As such, this research sets the stage for renewed interest within the scientific community, and offers a potential challenge to medical researchers to develop a human surgical procedure based on these successful animal experiments.
Abstract of Aldehyde-stabilized cryopreservation
We describe here a new cryobiological and neurobiological technique, aldehyde-stabilized cryopreservation (ASC), which demonstrates the relevance and utility of advanced cryopreservation science for the neurobiological research community. ASC is a new brain-banking technique designed to facilitate neuroanatomic research such as connectomics research, and has the unique ability to combine stable long term ice-free sample storage with excellent anatomical resolution. To demonstrate the feasibility of ASC, we perfuse-fixed rabbit and pig brains with a glutaraldehyde-based fixative, then slowly perfused increasing concentrations of ethylene glycol over several hours in a manner similar to techniques used for whole organ cryopreservation. Once 65% w/v ethylene glycol was reached, we vitrified brains at −135 °C for indefinite long-term storage. Vitrified brains were rewarmed and the cryoprotectant removed either by perfusion or gradual diffusion from brain slices. We evaluated ASC-processed brains by electron microscopy of multiple regions across the whole brain and by Focused Ion Beam Milling and Scanning Electron Microscopy (FIB-SEM) imaging of selected brain volumes. Preservation was uniformly excellent: processes were easily traceable and synapses were crisp in both species. Aldehyde-stabilized cryopreservation has many advantages over other brain-banking techniques: chemicals are delivered via perfusion, which enables easy scaling to brains of any size; vitrification ensures that the ultrastructure of the brain will not degrade even over very long storage times; and the cryoprotectant can be removed, yielding a perfusable aldehyde-preserved brain which is suitable for a wide variety of brain assays.
Schematic of proposed architecture of an implantable wireless-powered neural interface system that can provide power to implanted devices. Adding a transmitter chip could allow for neural signals to be transmitted via the antenna for external processing. (credit: Toyohashi University Of Technology)
A research team at Toyohashi University of Technology in Japan has fabricated an implanted wireless power transmission (WPT) device to deliver power to an implanted neural interface system, such as a brain-computer interface (BCI) device.
Described in an open-access paper in Sensors journal, the system avoids having to connect an implanted device to an external power source via wires through a hole in the skull, which can cause infections through the opening and risk of infection and leakage of the cerebrospinal fluid during long-term measurement. The system also allows for free-moving subjects, allowing for more natural behavior in experiments.
Photographs of fabricated flexible antenna and bonded CMOS rectifier chip with RF transformer (credit: Kenji Okabe et al./Sensors)
The researchers used a wafer-level packaging technique to integrate a silicon large-scale integration (LSI) chip in a thin (5 micrometers), flexible parylene film, using flip-chip (face-down) bonding to the film. The system includes a thin-film antenna and a rectifier to convert a radio-frequency signal to DC voltage (similar to how an RFID chip works). The entire system measures 27 mm × 5 mm, and the flexible film can conform to the surface of the brain.
Coventry University prof. Kevin Warwick turns on a light with a double-click of his finger, which triggers an implant in his arm (wired to a computer connected to the light). Adding an RF transmitter chip (and associated processing) to the Toyohashi system could similarly allow for controlling devices, but without wires. (credit: Kevin Warwick/element14)
The researchers plan to integrate additional functions, including amplifiers, analog-to-digital converters, signal processors, and a radio frequency circuit for transmitting (and receiving) data.
Tethered Braingate brain-computer interface for paralyzed patients (credit: Brown University)
This work is partially supported by Grants-in-Aid for Scientific Research, Young Scientists, and the Japan Society for the Promotion of Science.
element14 | Kevin Warwick’s BrainGate Implant
Abstract of Co-Design Method and Wafer-Level Packaging Technique of Thin-Film Flexible Antenna and Silicon CMOS Rectifier Chips for Wireless-Powered Neural Interface Systems
In this paper, a co-design method and a wafer-level packaging technique of a flexible antenna and a CMOS rectifier chip for use in a small-sized implantable system on the brain surface are proposed. The proposed co-design method optimizes the system architecture, and can help avoid the use of external matching components, resulting in the realization of a small-size system. In addition, the technique employed to assemble a silicon large-scale integration (LSI) chip on the very thin parylene film (5 μm) enables the integration of the rectifier circuits and the flexible antenna (rectenna). In the demonstration of wireless power transmission (WPT), the fabricated flexible rectenna achieved a maximum efficiency of 0.497% with a distance of 3 cm between antennas. In addition, WPT with radio waves allows a misalignment of 185% against antenna size, implying that the misalignment has a less effect on the WPT characteristics compared with electromagnetic induction.
Individual brain cells within a neural network are highlighted in this image obtained using a fluorescent imaging technique (credit: Sandra Kuhlman/CMU)
Carnegie Mellon University is embarking on a five-year, $12 million research effort to reverse-engineer the brain and “make computers think more like humans,” funded by the U.S. Intelligence Advanced Research Projects Activity (IARPA). The research is led by Tai Sing Lee, a professor in the Computer Science Department and the Center for the Neural Basis of Cognition (CNBC).
A “Human Genome Project” for the brain’s visual system
“MICrONS is similar in design and scope to the Human Genome Project, which first sequenced and mapped all human genes,” Lee said. “Its impact will likely be long-lasting and promises to be a game changer in neuroscience and artificial intelligence.”
The researchers will attempt to discover the principles and rules the brain’s visual system uses to process information. They believe this deeper understanding could serve as a springboard to revolutionize machine learning algorithms and computer vision.
In particular, the researchers seek to improve the performance of artificial neural networks — computational models for artificial intelligence inspired by the central nervous systems of animals. Interest in neural nets has recently undergone a resurgence thanks to growing computational power and datasets. Neural nets now are used in a wide variety of applications in which computers can learn to recognize faces, understand speech and handwriting, make decisions for self-driving cars, perform automated trading and detect financial fraud.
How neurons in one region of the visual cortex behave
“But today’s neural nets use algorithms that were essentially developed in the early 1980s,” Lee said. “Powerful as they are, they still aren’t nearly as efficient or powerful as those used by the human brain. For instance, to learn to recognize an object, a computer might need to be shown thousands of labeled examples and taught in a supervised manner, while a person would require only a handful and might not need supervision.”
To better understand the brain’s connections, Sandra Kuhlman, assistant professor of biological sciences at Carnegie Mellon and the CNBC, will use a technique called “two-photon calcium imaging microscopy” to record signaling of tens of thousands of individual neurons in mice as they process visual information, an unprecedented feat. In the past, only a single neuron, or tens of neurons, typically have been sampled in an experiment, she noted.
“By incorporating molecular sensors to monitor neural activity in combination with sophisticated optical methods, it is now possible to simultaneously track the neural dynamics of most, if not all, of the neurons within a brain region,” Kuhlman said. “As a result we will produce a massive dataset that will give us a detailed picture of how neurons in one region of the visual cortex behave.”
A multi-institution research team
Other collaborators are Alan Yuille, the Bloomberg Distinguished Professor of Cognitive Science and Computer Science at Johns Hopkins University, and another MICrONS team at the Wyss Institute for Biologically Inspired Engineering, led by George Church, professor of genetics at Harvard Medical School.
The Harvard-led team, working with investigators at Cold Spring Harbor Laboratory, MIT, and Columbia University, is developing revolutionary techniques to reconstruct the complete circuitry of the neurons recorded at CMU. The database, along with two other databases contributed by other MICrONS teams, unprecedented in scale, will be made publicly available for research groups all over the world.
In this MICrONS project, CMU researchers and their collaborators in other universities will use these massive databases to evaluate a number of computational and learning models as they improve their understanding of the brain’s computational principles and reverse-engineer the data to build better computer algorithms for learning and pattern recognition.
“The hope is that this knowledge will lead to the development of a new generation of machine learning algorithms that will allow AI machines to learn without supervision and from a few examples, which are hallmarks of human intelligence,” Lee said.
The CNBC is a collaborative center between Carnegie Mellon and the University of Pittsburgh. BrainHub is a neuroscience research initiative that brings together the university’s strengths in biology, computer science, psychology, statistics and engineering to foster research on understanding how the structure and activity of the brain give rise to complex behaviors.
The MICrONS team at CMU allso includes Abhinav Gupta, assistant professor of robotics; Gary Miller, professor of computer science; Rob Kass, professor of statistics and machine learning and interim co-director of the CNBC; Byron Yu, associate professor of electrical and computer engineering and biomedical engineering and the CNBC; Steve Chase, assistant professor of biomedical engineering and the CNBC; and Ruslan Salakhutdinov, one of the co-creators of the deep belief network, a new model of machine learning that was inspired by recurrent connections in the brain, who will join CMU as an assistant professor of machine learning in the fall.
Other members of the team include Brent Doiron, associate professor of mathematics at Pitt, and Spencer Smith, assistant professor of neuroscience and neuro-engineering at the University of North Carolina.
Not all machine-intelligence experts are on board with reverse-engineering the brain. In a Facebook post today, Yann LeCun, Director of AI Research at Facebook and a professor at New York University, asked the question in a recent lecture, “Should we copy the brain to build intelligent machines?” “My answer was ‘no, because we need to understand the underlying principles of intelligence to know what to copy. But we should draw inspiration from biology.’”
As a test of penetrating the blood-brain barrier using a harmless virus (AAV-PHP.B), green fluorescent protein (GFP) was created in the mouse cerebellum via delivery of the GFP gene via the virus. Purkinje neurons (shown in magenta) are responsible for sending signals out of the cerebellum. (credit: Ben Deverman and the Gradinaru laboratory/Caltech)
Caltech biologists have modified a harmless virus to allow it to enter the adult mouse brain through the bloodstream and deliver genes to cells of the nervous system.
The modified virus could lead to novel therapeutics to address diseases such as Alzheimer’s and Huntington’s, help researchers map the brain, and target cells in other organs, according to Ben Deverman, a senior research scientist at Caltech and lead author of a paper describing the work in the February 1 online publication of the journal Nature Biotechnology.
A viral penetration strategy
The blood-brain barrier (BBB) allows the body to keep pathogens and potentially harmful chemicals circulating in the blood from entering the brain and spinal cord. But that also makes it nearly impossible for many drugs and other molecules to be delivered to the brain via the bloodstream.
One variant of the AAV virus (credit: Jazzlw/Creative Commons)
The small, harmless adeno-associated virus (AAV) has been used to transport specific genes into the nuclei of cells; once there, the genes can be expressed (translated from DNA into proteins). AAVs can carry functional copies of genes to replace mutated forms present in individuals with genetic diseases. AAVs can also be used to deliver genes that provide instructions for generating molecules such as antibodies or fluorescent proteins that help researchers study, identify, and track certain cells.
Unfortunately, the blood-brain barrier has made it difficult to deliver AAVs and their genetic cargo to the central nervous system. So researchers have mostly relied on surgical injections, which deliver high concentrations of the virus at the injection site but little to the outlying areas.
That method is also extremely invasive. “One has to drill a hole through skull, then pierce tissue with a needle to the injection site,” explains Viviana Gradinaru, assistant professor of biology and biological engineering at Caltech and senior author on the paper. “The deeper the injection, the higher the risk of hemorrhage.”
Guradinaru says delivering via the BBB would avoid those problems, and “many disorders are not tightly localized. Neurodegenerative disorders like Huntington’s disease affect very large brain areas. Also, many complex behaviors are mediated by distributed interacting networks.”
Testing millions of self-replicating viral variants
In 2009, a group led by Brian Kaspar of Ohio State University published a paper, also in Nature Biotechnology, showing that an AAV strain called AAV9 injected into the bloodstream could make its way into the brain — but it was only efficient when used in neonatal (infant) mice. “The big challenge was how do we achieve the same efficiency in an adult,” says Gradinaru.
To do that efficiently, the researchers developed a high-throughput selection assay called CREATE (Cre REcombinase-based AAV Targeted Evolution) that allowed them to test millions of self-replicating viral variants of AAV in vivo simultaneously and to identify those that were best at entering the brain and delivering genes to a specific class of brain cells known as astrocytes.*
They also used PARS CLARITY, a technique developed previously in the Gradinaru lab to make normally opaque mammalian tissues transparent, allows organs to be examined without the laborious task of making thin slide-mounted sections. This allows researchers to more quickly screen the viral vectors for those that best target the cells and organs of interest.**
They narrowed it down to one variant, which they named AAV-PHP.B — it delivers genes to the brain and spinal cord efficiently through the vasculature into the brain (and to other tissues in the body).
In collaboration with colleagues from Stanford University, the researchers also showed that AAV-PHP.B is better than AAV9 at delivering genes to human neurons and glia.
Using a new selection method, Caltech researchers have evolved the protein shell of a harmless virus, AAV9, so that it can more efficiently cross the blood brain barrier and deliver genes, such as the green fluorescent protein (GFP), to cells throughout the central nervous system. Here, GFP expression in naturally occurring AAV9 (left) can be seen distributed sparsely throughout the brain. The modified vector, AAV-PHP.B (right), provides more efficient GFP expression. (credit: Ben Deverman and the Gradinaru laboratory/Caltech)
The researchers hope to begin testing AAV-PHP.B’s ability to deliver potentially therapeutic genes in disease models. They are also working to further evolve the virus to make even better performing variants and to produce variants that target certain cell types with more specificity.
The Beckman Institute at Caltech recently opened a resource center called CLOVER (CLARITY, Optogenetics, and Vector Engineering Research Center) to support such research efforts involving tissue clearing and imaging, optogenetic studies, and custom gene-delivery vehicle development. Deverman is the center’s director, and Gradinaru is the principal investigator.
The work was supported by funding from the Hereditary Disease Foundation and the Caltech-City of Hope Biomedical Initiative, a National Institutes of Health (NIH) Director’s New Innovator Award, the NIH’s National Institute of Aging and National Institute of Mental Health, the Beckman Institute, and the Gordon and Betty Moore Foundation.
* They started with the AAV9 virus and modified a gene fragment that codes for a small loop on the surface of the capsid — the protein shell of the virus that envelops all of the virus’ genetic material. Using a common amplification technique, known as polymerase chain reaction (PCR), they created millions of viral variants. Each variant carried within it the genetic instructions to produce more capsids like itself.
Then they used their novel selection process to determine which variants most effectively delivered genes to astrocytes in the brain. Importantly, the new process relies on strategically positioning the gene encoding the capsid variants on the DNA strand between two short sequences of DNA, known as lox sites. These sites are recognized by an enzyme called Cre recombinase, which binds to them and inverts the genetic sequence between them. By injecting the modified viruses into transgenic mice that only express Cre recombinase in astrocytes, the researchers knew that any sequences flagged by the lox site inversion had successfully transferred their genetic cargo to the target cell type—here, astrocytes.
After one week, the researchers isolated DNA from brain and spinal cord tissue, and amplified the flagged sequences, thereby recovering only the variants that had entered astrocytes.
Next, they took those sequences and inserted them back into the modified viral genome to create a new library that could be injected into the same type of transgenic mice. After only two such rounds of injection and amplification, a handful of variants emerged as those that were best at crossing the blood-brain barrier and entering astrocytes.
“We went from millions of viruses to a handful of testable, potentially useful hits that we could go through systematically and see which ones emerged with desirable properties,” says Gradinaru.
Through this selection process, the researchers identified a variant dubbed AAV-PHP.B as a top performer. To test AAV-PHP.B, the researchers used it to deliver a gene that codes for a protein that glows green, making it easy to visualize which cells were expressing it. They injected the AAV-PHP.B or AAV9 (as a control) into different adult mice and after three weeks used the amount of green fluorescence to assess the efficacy with which the viruses entered the brain, the spinal cord, and the retina.
“We could see that AAV-PHP.B was expressed throughout the adult central nervous system with high efficiency in most cell types,” says Gradinaru. Indeed, compared to AAV9, AAV-PHP.B delivers genes to the brain and spinal cord at least 40 times more efficiently.
“What provides most of AAV-PHP.B’s benefit is its increased ability to get through the vasculature into the brain,” says Deverman. “Once there, many AAVs, including AAV9 are quite good at delivering genes to neurons and glia.”
** Making tissues transparent
Gradinaru notes that since AAV-PHP.B is delivered through the bloodstream, it reaches other parts of the body. “Although in this study we were focused on the brain, we were also able to use whole-body tissue clearing to look at its biodistribution throughout the body,” she says.
“In this case, the priority was to express the gene in the brain, but we can see by using whole-body clearing that you can actually have expression in many other organs and even in the peripheral nerves,” explains Gradinaru. “By making tissues transparent and looking through them, we can obtain more information about these viruses and identify targets that we might overlook otherwise.”
The biologists conducted follow-up studies up to a year after the initial injections and found that the protein continued to be expressed efficiently. Such long-term expression is important for gene therapy studies in humans.
Deverman says that the CREATE system could indeed be applied to develop AAVs capable of delivering genes specifically to many different cell types. “There are hundreds of different Cre transgenic lines available,” he says. “Researchers have put Cre recombinase under the control of gene regulatory elements so that it is only made in certain cell types. That means that regardless of whether your objective is to target liver cells or a particular type of neuron, you can almost always find a mouse that has Cre recombinase expressed in those cells.”
“The CREATE system gave us a good hit early on, but we are excited about the future potential of using this approach to generate viruses that have very good cell-type specificity in different organisms, especially the less genetically tractable ones,” says Gradinaru. “This is just the first step. We can take these tools and concepts in many exciting directions to further enhance this work, and we—with the Beckman Institute and collaborators—are ready to pursue those possibilities.”
Ben Deverman and the Gradinaru laboratory/Caltech and Nature Biotechnology | Green fluorescent protein (GFP) expression in the spinal cord of an adult mouse that received an intravenous injection of the modified virus, AAV-PHP.B, carrying the gene for GFP. The spinal cord sample was rendered transparent for imaging using a whole-body tissue clearing method called PARS CLARITY previously developed by the Gradinaru lab.
Abstract of Cre-dependent selection yields AAV variants for widespread gene transfer to the adult brain
Recombinant adeno-associated viruses (rAAVs) are commonly used vehicles for in vivogene transfer. However, the tropism repertoire of naturally occurring AAVs is limited, prompting a search for novel AAV capsids with desired characteristics. Here we describe a capsid selection method, called Cre recombination–based AAV targeted evolution (CREATE), that enables the development of AAV capsids that more efficiently transduce defined Cre-expressing cell populations in vivo. We use CREATE to generate AAV variants that efficiently and widely transduce the adult mouse central nervous system (CNS) after intravenous injection. One variant, AAV-PHP.B, transfers genes throughout the CNS with an efficiency that is at least 40-fold greater than that of the current standard, AAV9, and transduces the majority of astrocytes and neurons across multiple CNS regions. In vitro, it transduces human neurons and astrocytes more efficiently than does AAV9, demonstrating the potential of CREATE to produce customized AAV vectors for biomedical applications.
This 3D gel model of a smooth fetal brain is coated with a thin layer of elastomer gel and immersed in solvent. The gel absorbs the solvent, causing it to swell relative to the deeper regions. Within minutes, the resulting compression leads to the formation of folds similar in size and shape to real brains. (credit: Mahadevan Lab/Harvard SEAS)
Folded brains likely evolved to fit a large cortex into a small volume, with the added benefit of reducing neuronal wiring length and improving cognitive function. But how does the brain fold?
“We found that we could mimic cortical folding using a very simple physical principle and get results qualitatively similar to what we see in real fetal brains,” said L. Mahadevan, the Lola England de Valpine Professor of Applied Mathematics, Organismic and Evolutionary Biology, and Physics.
The number, size, shape, and position of neuronal cells present during brain growth all lead to the expansion of the gray matter (cortex) relative to the underlying white matter. This puts the cortex under compression, leading in turn to a mechanical instability that causes it to crease locally.
“This simple evolutionary innovation, with iterations and variations, allows for a large cortex to be packed into a small volume, and is likely the dominant cause behind brain folding, known as gyrification,” said Mahadevan, who is also a core faculty member of the Wyss Institute for Biologically Inspired Engineering and a member of the Kavli Institute for Bionano Science and Technology, both at Harvard University.
Mahadevan’s previous research found that the growth differential between the brain’s outer cortex and the soft tissue underneath explains the variations in the folding patterns across organisms in terms of just two parameters: the relative size of the brain and the relative expansion of the cortex.
Testing a 3D gel model
Left: All notable gyri as identified on a real fetal brain. Right: Analogous regions shown in a simulated brain driven by relative constrained growth. In both cases, the coloring is based on visual identification of the major gyri. (credit: adapted from P. D. Griffiths et al./Atlas of fetal and neonatal brain MR imaging, with permission from Elsevier; and Tuomas Tallinen et al./Nature Physics)
Building on this, the team collaborated with neuroanatomists and radiologists in France and directly tested this theory using data from human fetuses. The team made a 3D gel model of a smooth fetal brain based on MRI images. The model’s surface was coated with a thin layer of elastomer gel, as an analog of the cortex. To mimic cortical expansion, the gel brain was immersed in a solvent that is absorbed by the outer layer, causing it to swell relative to the deeper regions. Within minutes of being immersed in liquid solvent, the resulting compression led to the formation of folds similar in size and shape to real brains.
The extent of the similarities surprised even the researchers. “When I put the model into the solvent, I knew there should be folding but I never expected that kind of close pattern compared to human brain,” said Jun Young Chung, a postdoctoral fellow and co-first author of the paper. “It looks like a real brain.”
The key to those similarities lies in the unique shape of the human brain. “The geometry of the brain is really important because it serves to orient the folds in certain directions,” said Chung. “Our model, which has the same large-scale geometry and curvature as a human brain, leads to the formation of folds that matches those seen in real fetal brains quite well.
“Brains are not exactly the same from one human to another, but we should all have the same major folds in order to be healthy,” said Chung. “Our research shows that if a part of the brain does not grow properly, or if the global geometry is disrupted, we may not have the major folds in the right place, which may cause dysfunction in the brain.”
The distinctive folds of the human brain are not present in most animals — only in a handful of species, including some primates, dolphins, elephants, and pigs. In humans, folding begins in fetal brains around the 20th week of gestation and is completed only when the child is about 18 months old.
Scientists at Jyvaskyla University in Finland, the Institut Mines-Télécom in France, and the University of Aix-Marseille, France were also involved in the research, which was supported by the Academy of Finland, the Wyss Institute for Biologically Inspired Engineering, and fellowships from the MacArthur Foundation and the Radcliffe Institute.
Abstract of On the growth and form of cortical convolutions
The rapid growth of the human cortex during development is accompanied by the folding of the brain into a highly convoluted structure. Recent studies have focused on the genetic and cellular regulation of cortical growth, but understanding the formation of the gyral and sulcal convolutions also requires consideration of the geometry and physical shaping of the growing brain. To study this, we use magnetic resonance images to build a 3D-printed layered gel mimic of the developing smooth fetal brain; when immersed in a solvent, the outer layer swells relative to the core, mimicking cortical growth. This relative growth puts the outer layer into mechanical compression and leads to sulci and gyri similar to those in fetal brains. Starting with the same initial geometry, we also build numerical simulations of the brain modelled as a soft tissue with a growing cortex, and show that this also produces the characteristic patterns of convolutions over a realistic developmental course. All together, our results show that although many molecular determinants control the tangential expansion of the cortex, the size, shape, placement and orientation of the folds arise through iterations and variations of an elementary mechanical instability modulated by early fetal brain geometry.
