pen Life Lab
Sources / summaries / project relevance
Explore the papers informing the experiments and evolving design. Each summary separates the paper’s findings from possible project implications.
269 references
Alphabetical by titlePresents fully open dense language models, releasing training data, code, logs, and intermediate checkpoints. Architectural and training changes target stability and efficiency; a late-stage specialized data mixture and subsequent instruction training improve evaluated capabilities.
Separate glutamate uncaging, postsynaptic spiking and optogenetic dopamine manipulations in mouse nucleus-accumbens D1 medium spiny neurons identify a short timing window for dopamine-promoted spine enlargement after conditioning. Activity-specific local processes interact with dopamine/cyclic-AMP/PKA/CaMKII mechanisms. Dopamine did not simply increase the calcium transient in the reported tests; broader PKA and local CaMKII-related effects were distinguishable.
Simultaneous voltage and calcium imaging in behaving-mouse CA2 pyramidal cells found largely output-linked dendritic calcium, with complex spikes preferentially recruiting distal branches and isolated local voltage/calcium events rare. Branch-specific calcium could vary more than voltage, and a compact channel-inspired transfer model predicted amplitudes. This challenges inference of autonomous dendritic computation from calcium alone under these conditions.
Introduces a self-referential weight matrix that generates signals used to modify its own matrix during execution. Outer-product and delta-rule operations make the construction finite and scalable; experiments evaluate few-shot learning and multitask reinforcement learning.
This author synthesis relates earlier primate dopamine recordings to temporal-difference prediction theory. Selected evidence includes responses to unexpected appetitive outcomes, transfer toward predictive cues after training, and decreased firing at the expected time of omitted reward. The computational account adds a prediction target, temporal representation and learning assumptions; it is not an experiment demonstrating multilayer chain-rule gradients.
A delay-reconstruction estimator tracks nearby trajectory segments with temporal-neighbor exclusion and fits a slope to average logarithmic divergence. Tests explore noise, embedding and quasiperiodic/stochastic controls. A bounded trajectory produces eventual distance saturation; short portions of nonchaotic curves may resemble positive scaling regions. The method supports conditional evidence with diagnostic plots rather than blind reliance on a fitted slope.
The paper formulates squid-axon membrane current using capacitance, voltage/time-dependent sodium and potassium conductances and leak current. The targeted conduction section compares a calculated propagated action potential and speed with measured records. Sodium conductance dominates the rapid rise, while delayed potassium conductance and sodium inactivation shape recovery and refractoriness. Ionic-flux comparisons include discrepancies with tracer measurements. The accommodation discussion explicitly says neither direct measurements nor corresponding model calculations were made there; it offers qualitative arguments. Its oscillations section describes heavily damped returns near rest and small constant-current responses, not a general proof of sustained irregular dynamics.
Proposes a connection between organizational closure and open-endedness using multidimensional mappings. A Game of Life toy analysis illustrates the proposed concept of open-ended organizational closure, while the authors identify limits of that demonstration.
Wanda prunes pretrained language-model weights using weight magnitude multiplied by corresponding input activation magnitude. It avoids retraining and expensive second-order reconstruction, and evaluates how much language-model performance remains after pruning.
Constructs one-way and totalistic normal forms for one-dimensional cellular automata, then encodes them in a small universal cellular automaton with fourteen states. The result concerns simulation of other computations through suitable initial encodings.
Reports activity-dependent regulation of excitatory synaptic strength in cultured neocortical neurons. Changes in overall activity induce compensatory changes in quantal amplitude, supporting a homeostatic scaling mechanism distinct from strengthening only particular active contacts.
Uses evolving digital organisms to investigate diversity under resource competition. Mixed limiting resources produce frequency-dependent selection and distinct organism groups; maximum richness can occur at intermediate productivity even in a spatially homogeneous environment.
Examines repeated training on model-generated data and finds degradation of the represented distribution, including loss of rare content. The analysis and experiments connect recursive replacement of real data with accumulating defects in later model generations.
Develops a quasi-two-dimensional microfluidic model of alkaline vents. Flow and ion conditions alter mineral precipitation morphology; imaging shows that some structures support microscale pH gradients and particle accumulation.
Images fluorescent AMPA receptors in rat hippocampal slices during long-term potentiation. Most incorporated receptors arrive by lateral diffusion from the spine surface; subsequent exocytosis primarily replenishes dendritic surface pools for later plasticity.
Models AMPA-receptor trafficking in a three-dimensional spiny dendritic segment. Strengthening or weakening one synapse changes receptor availability at nearby unstimulated synapses, producing compensatory heterosynaptic effects and altering later plasticity.
An editorial overview organizes open-ended-evolution research and the associated special issues. It surveys approaches, conceptual distinctions, and remaining difficulties, emphasizing that existing artificial systems have not matched biological evolutionary open-endedness.
Studies analog recurrent neural computation with a fixed network structure. Under idealized real-valued parameters, the model's computational power and resource bounds relate to nonuniform circuits and can exceed ordinary digital computation under the stated assumptions.
The paper analyzes the geometry of input-history representations in driven echo-state networks and proposes using fractal dimension when choosing recurrent/input gains. Binary inputs and two-neuron reservoirs make the state sets visible. Arithmetic encoding supplies an analogy for dense, non-overlapping representations under specified input probabilities. One illustrated configuration gives box-counting slopes about1.9066 versus1.2670 at different input gains. Simulations for the state plots use100 sequences of roughly one million steps each. SVM support-vector counts probe separation complexity. The proposed dimension/encoding criterion and refined echo-state property remain preliminary; this is not a broad matched demonstration that maximizing dimension improves task performance.
Proves that finite sums of affine inputs passed through suitable sigmoidal functions can approximate continuous functions on a compact domain. The theorem establishes representational capacity of a single-hidden-layer feedforward architecture.
Establishes universal approximation results for continuous sequence-to-sequence functions by Transformers. Attention builds contextual mappings, while positional encodings remove the restriction to permutation-equivariant functions under the theorem's conditions.
Random synchronous threshold networks are analyzed using common-input Hamming-distance growth and bit-flip sensitivity. Delayed parity and random Boolean-function readouts perform well near a critical line. A local sensitivity-based scaling rule approaches that line and improves measured capability. Readouts are trained anew with recurrent weights frozen at each evaluation; this is not unrestricted simultaneous online learning.
Introduces the Transformer encoder-decoder architecture, replacing recurrent sequence processing with multi-head attention, feedforward layers, and positional information. Machine-translation experiments demonstrate strong task performance and greater training parallelism.
Tests whether attention weights explain predictions in neural language tasks. Weak agreement with gradient-based importance and alternative attention distributions giving similar predictions challenge the use of attention maps as direct causal explanations.
Challenges broad claims that attention cannot explain predictions and proposes stronger diagnostic tests. Uniform-weight baselines, variation across seeds, frozen attention, and adversarial evaluation examine whether attention carries model-specific explanatory information.
Identifies reflexively autocatalytic food-generated networks within microbial metabolism. Networks supplied with suitable small-molecule catalysts can generate key biochemical building blocks, offering a proposed connection between simpler autocatalytic systems and metabolism.
Extends discussion of autocatalytic sets to include boundaries generated and maintained by internal reaction networks. The proposed formalization connects RAF theory with chemoton, autopoietic, and related models and allows higher-level sets to form.
Argues that autopoiesis alone needs an account of adaptivity to explain regulation relative to viability. The conceptual analysis develops links among self-maintenance, intrinsic values, sense-making, and agency.
Describes Avida, a platform for controlled evolution of self-replicating computer programs. It explains the execution substrate, experimental setup, measurement tools, and analysis practices used to investigate evolutionary mechanisms.
AWQ compresses language models through low-bit weight quantization guided by activation-sensitive channels. Equivalent channel scaling protects influential weights while retaining a hardware-friendly format; evaluations examine accuracy and deployment efficiency.
Backpropamine adds differentiable neuromodulation to plastic neural networks. Network-generated signals regulate Hebbian changes during execution, while gradient-based outer training learns useful parameters; supervised and reinforcement-learning tasks test the approach.
Investigates how bacteria track short-lived chemical gradients. Quantitative analysis of chemotactic responses connects sensory precision and motility to rapid aggregation around changing resource patches.
BERT pretrains bidirectional Transformer representations using masked text and then adapts them to downstream language tasks. Joint access to left and right context improves the evaluated understanding tasks relative to earlier pretraining baselines.
Uses a detailed layer-five pyramidal-cell model to study synaptic conditions generating bursts. Nonlinear dendritic events interact with excitation and inhibition, allowing distinct input streams to alter the occurrence and properties of output bursts.
Analyzes the order-to-chaos transition in Boolean networks built from canalizing functions. The proposed criterion accounts for nonergodic activity patterns and is compared with numerical simulations.
Argues that formation of higher-order entities expands the space available to evolution. Hash Chemistry illustrates the proposal using variable-size entities and a hash-based fitness evaluator, with continued appearance of new replicating combinations.
Introduces CAX, a modular JAX library for hardware-accelerated discrete and continuous cellular automata. Benchmarks and applications demonstrate support for several dimensions and models, including neural cellular automata.
In juvenile rat entorhinal slices with ionotropic synaptic activity blocked, riluzole reduces native persistent sodium conductance and theta-band perithreshold power. Dynamic clamp then inserts virtual persistent sodium conductance with matched deterministic or stochastic mean gating. Stochastic replacement better restores perithreshold power (14 cells in Figure 2) and phase-response behavior than deterministic replacement. The implementation uses a fixed 1,200-channel approximation and 13,333-Hz updates. Phase reliability responses vary across twelve cells: channel noise can add jitter or support responsiveness rather than uniformly improving precision. Partial channel blockade, exclusively somatic insertion, fixed virtual channel count and uncertain spatial conductance leave explicit limits. The discussion entertains damped/unstable oscillatory trajectories kicked by noise; it does not establish deterministic chaos.
Profiles transcriptomic and epigenomic features across human, marmoset, and mouse motor cortex. The study identifies broadly conserved cell-type organization alongside species-specific proportions, molecular features, and regulatory differences.
Investigates cellular automata near transitions between ordered and chaotic behavior. The paper argues that these regimes can support information storage, transmission, and modification, connecting computational phenomena with dynamical phase transitions.
Uses single-nucleus RNA sequencing to classify human cortical cells and compare homologous mouse types. Broad cellular organization is conserved, but proportions, gene expression, layer distribution, and morphology differ substantially.
Trains assistants using self-critiques, revisions, and AI-generated preferences guided by a human-written constitution. Supervised and reinforcement-learning stages shift behavior without human harmlessness labels for each example.
Maps sensory and pontine projections at synaptic resolution and finds convergence onto individual cerebellar granule cells. This supports multimodal integration within cells previously discussed as potentially unimodal relays.
Corrects a data error in the earlier hippocampal prediction-error study. The notice revises numerical results and supporting material; the authors state that the principal findings and conclusions remain unchanged.
Demonstrates growth and division of model fatty-acid vesicles. Feeding multilamellar vesicles produces elongated forms under limited solute permeability; physical division offers a route to repeated membrane cycles with retained contents.
Combines phylogenetic and fossil evidence to investigate cyanobacterial origins and diversification around Earth's oxygenation. It discusses early multicellularity and why oxygen production could precede atmospheric oxygen accumulation.
Shows that L2 penalties and weight decay behave differently in adaptive optimizers such as Adam. Decoupling decay from the loss-gradient update improves evaluated generalization and makes decay tuning less dependent on learning-rate choice.
Measures duplication and train-test overlap in language-model corpora. Exact and near-duplicate removal reduces memorized output and can improve learning efficiency while making evaluation less contaminated.
Deep equilibrium models solve for fixed points of a shared transformation rather than explicitly storing a deep stack of intermediate layers. Implicit differentiation enables training with memory independent of effective depth; language experiments evaluate the approach.
A simple Deep GA evolves fixed-topology neural parameters using truncation selection, additive Gaussian mutations and elitism, omitting crossover for simplicity. A network with more than four million parameters achieves competitive results on selected Atari benchmarks. The paper discusses unequal frame budgets and evaluation-start conventions among some external comparisons. This establishes a useful mutation-only baseline, but contains no ablation showing crossover would worsen that GA.
Demonstrates reinforcement learning of language-model reasoning with verifiable task rewards, without requiring human reasoning trajectories in the initial pure-RL setup. Additional training stages produce the reported reasoning model and evaluate mathematical and coding tasks.
DeepSeek-V2 combines sparse mixture-of-experts computation with latent compression of attention's key-value cache. The report evaluates training economy and inference throughput while describing pretraining and post-training.
DeepSeek-V3 develops a large mixture-of-experts model with latent attention, load-balancing changes, and multi-token prediction. The report describes pretraining and post-training and presents task benchmarks alongside training-resource accounting.
DeepSeek-V3.2 combines sparse attention, scaled reinforcement learning, and synthesized agent tasks. The report argues these changes improve long-context efficiency and evaluated reasoning and tool-use performance.
DeepSeekMath combines mathematical-data selection, continued pretraining, and Group Relative Policy Optimization. Experiments evaluate mathematical reasoning and compare training and sampling choices.
Develops a definition of agency around self-defined individuality, active contribution to interaction, and regulation relative to norms. Minimal protocellular models motivate a generative organizational account.
Derives activation-dependent initialization for deep rectifier networks and evaluates PReLU image classifiers. The forward calculation uses input second moments because rectifier outputs are not centered. Independence and symmetric preactivations yield variance 2/fan-in for ReLU; changing the negative slope changes the factor.
Records active dendritic properties in human cortical neurons and identifies graded calcium-mediated dendritic action potentials. The study argues that these nonlinear events enable individual neurons to classify inputs beyond a simple linear threshold.
A compartmental cortical-circuit model represents apical error as top-down feedback minus predictive interneuron inhibition. A self-predicting state cancels expected feedback, and output nudging unmasks an instructive residual. Approximation to backpropagation requires suitable small-feedback and predictive-state assumptions; symmetric, random and learned feedback cases differ. Continuous-time regression avoids mandatory phases, while classification demonstrations use computational shortcuts.
Makes Hebbian plasticity differentiable so gradient-based outer training can optimize plastic-network parameters. Recurrent networks then change connection state during new tasks, with memory, meta-learning, and reinforcement-learning demonstrations.
Investigates the slow afterhyperpolarization in rat hippocampal neurons near physiological temperature. Calcium-activated potassium currents and sodium-potassium pumps contribute, with pumps dominating some short-spike-train responses.
Simulates irreversible particle aggregation with rotational diffusion and attractive or repulsive interactions. Aggregate structure and aggregation times depend on interaction conditions; repulsion can produce more compact structures.
Derives Direct Preference Optimization from a reward-model parameterization, yielding a direct language-model objective for preference pairs. It avoids a separate reward-model-and-RL pipeline and evaluates preference alignment.
Studies chemically driven self-assembly leading to adaptive fibril behavior and division. Continued energy-consuming chemistry couples changes in assembled structure to how the system processes its drive.
A theoretical perspective relates nonequilibrium dissipation, irreversibility, and driven self-assembly. It discusses how externally forced systems can acquire structures suited to absorbing and dissipating the supplied work.
Transfers predictive information from an ensemble or large teacher to a smaller model using softened output distributions. Experiments show that a compact student can retain useful knowledge beyond training only on hard labels.
Introduces Dolma, a large openly documented text corpus, with source mixtures, filtering, and construction details. Intermediate-data analyses and training experiments make data choices more inspectable.
Tests continued pretraining on domain and task text before supervised adaptation. Across several domains and classification tasks, additional relevant unlabeled data improves the evaluated downstream results.
DoRA separates weight magnitude and direction, using low-rank updates for the directional component. Comparisons with LoRA and full fine-tuning evaluate adaptation accuracy and stability without additional inference overhead after merging.
Uses mean-field and random-matrix analyses of recurrent networks at initialization. Vanilla networks have restrictive conditions for long signal propagation; gating broadens useful regimes. Nonlinear Jacobians depend on activation derivatives, so an orthogonal weight matrix alone does not establish dynamical isometry.
Uses nuclear-bomb-derived carbon dating of DNA to estimate adult human hippocampal cell turnover. A population model supports continued neuron generation within a renewing fraction, with age-related changes.
Tests three entropy-production hypotheses in food-web models. Living states increase production relative to abiotic baselines in the tested cases, but maximum-production state selection and monotonic gradient-response claims fail with multiple trophic levels.
Combines tensor, pipeline, and data parallelism for training very large language models across GPU clusters. An interleaved pipeline schedule reduces idle time, with throughput measurements illustrating scaling constraints.
PagedAttention stores attention caches in noncontiguous blocks and permits sharing where appropriate. The vLLM serving implementation reduces fragmentation and redundant cache storage, improving evaluated throughput under comparable latency.
Shows synthetic self-replicators recruiting a photocatalytic cofactor that converts environmental molecules into replication precursors. Light-driven chemistry creates positive feedback between the replicator and its own material supply.
Uses molybdenum-isotope signatures in ancient rocks to infer interaction with manganese oxides and local oxygen production before the Great Oxidation Event. The interpretation places oxygenic photosynthesis earlier than sustained atmospheric oxygen accumulation.
Evolves virtual prey that construct physical structures to avoid a predefined predator. A two-dimensional physics environment and comparisons with ecological inheritance investigate how retained structures influence adaptation and structural diversity.
Finds sodium-channel-related genes in a single-celled animal relative and compares channel sequences across early-diverging animals. Phylogenetic and molecular evidence suggests these channels predate nervous systems.
Evolves cellular automata for density classification and analyzes successive computational strategies. The study identifies innovation episodes and search impediments, including symmetry breaking that gives short-term gains while blocking stronger solutions.
NEAT aligns connection genes by historical innovation identifiers, inherits matching genes from either parent and unmatched genes primarily from the fitter parent, with speciation and incremental structural growth. On double-pole balancing with velocity inputs, its crossover ablation averages 3,600 versus 5,557 evaluations across 120 runs per condition, p=0.001, with zero reported failures in these two conditions. The approximately 35.2% reduction is task/system-specific.
Analyzes gradient-learning dynamics in deep linear networks. Orthogonal initialization can preserve singular values through linear products, whereas average norm preservation by Gaussian matrices can conceal poorly conditioned directions. The nonlinear discussion adds activation-dependent qualifications.
T5 casts diverse language tasks into a common text-to-text format. Systematic comparisons of architectures, pretraining objectives, data, and transfer procedures inform the resulting pretrained models.
FlashAttention-2 improves exact attention implementation through better work partitioning and reduced overhead. The analysis focuses on GPU occupancy, inter-warp communication, and balancing attention against efficient matrix multiplication.
FlashAttention tiles exact attention to reduce transfers between GPU main memory and on-chip storage. It avoids materializing large intermediate attention arrays, improving memory use and measured runtime without the usual attention approximation.
Defines unbounded evolution and innovation relative to recurrence and trajectories attainable under fixed rules. Dynamical-system examples explore how time-dependent rules can satisfy these proposed open-endedness criteria.
FractalNet recursively expands a feedforward convolutional graph so short and long transformed paths join by elementwise means. Local/global drop-path makes individual columns learn usable predictors. On CIFAR-100 without augmentation the unregularized full model reports35.34% error and the regularized model28.20%; the regularized configuration also uses dropout. On ImageNet the34-layer model reports24.12% top-1 error against24.19% for the cited34-layer ResNet configuration. A plain40-layer CIFAR-100++ network reports38.84% error, a40-layer column trained inside the fractal graph31.75%, and the full graph27.40%. These support trainability and a multiscale training-scaffold interpretation. They do not isolate a universal benefit of self-similarity. Increasing depth from80 to160 raises error from26.46% to27.38% in the separate reduced-width comparison.
Releases co-registered mouse visual-cortex activity and electron-microscopy connectivity data. Calcium responses and detailed reconstructions connect functional observations with extensive structural wiring.
GaLore projects gradients into low-rank subspaces to reduce optimizer-state memory while permitting updates of full model weights. Pretraining and fine-tuning experiments evaluate accuracy and memory tradeoffs.
Gated DeltaNet combines adaptive memory gating with delta-rule updates in a linear recurrent architecture. Gating enables forgetting while targeted updates manage associations; a parallel algorithm supports hardware-efficient training.
Fits generalized leaky integrate-and-fire models of increasing complexity to recorded neurons. More complex point models better predict held-out spike timing, and parameter patterns help distinguish broad cell classes.
FORCE learning modifies readout/feedback or recurrent weights to organize initially chaotic rate-network activity into target patterns while feedback remains active. Rapid recursive least-squares error reduction keeps the feedback sufficiently controlled; low-amplitude targets can fail when chaos persists. The authors discuss limitations of explicit target/error signals, fast plasticity and correlation-aware updates. Initial chaos is not shown to be necessary for the learned function.
Selects fruit flies for a heat-induced wing phenotype and observes increased expression without the inducing treatment after selection. The experiments illustrate genetic assimilation of an initially environmentally elicited character.
Genetic manipulations separate innate and experience-dependent courtship potential in fruit flies. Males lacking one male-specific regulator can acquire persistent courtship through social experience when another regulator is present.
Reviews genotype-to-phenotype mapping and critiques the blueprint metaphor. Developmental processes, environmental context, and nonlinear interactions mediate the transformation from inherited information to realized traits.
A broad synthesis organizes neural architectures through geometric priors such as symmetry, equivariance, stability, and scale separation. It connects grids, graphs, groups, manifolds, and related domains to shared design principles.
Estimates radiative entropy production from satellite-derived cloud properties and atmospheric profiles. Shortwave absorption, longwave emission, and a steady-state balance are used to infer irreversible contributions within Earth's system.
Compares gated linear-unit variants in Transformer feedforward layers. Multiplicative projections with alternative nonlinearities improve evaluated pretraining and downstream language-task performance over conventional activations.
GPTQ performs post-training weight quantization using approximate second-order information. It compresses large pretrained Transformers to low bit widths while measuring retained language-model quality and implementation cost.
Grouped-query attention shares key-value heads among query-head groups, balancing cache cost and expressiveness. The paper also adapts existing multi-head checkpoints using a limited additional training budget.
Optogenetic manipulation at the worm neuromuscular junction shows graded neurotransmitter release. Responses scale with depolarization and include tonic release modulated by hyperpolarization, rather than relying solely on discrete spikes.
Graph Attention Networks weight neighboring features through learned masked attention. The architecture supports graph-local processing without requiring fixed global graph structure and is evaluated on node-classification benchmarks.
Trains shared local neural cellular-automaton rules to grow target patterns from seeds. Persistence and damage training produce stronger long-term stability and regeneration, illustrating learned distributed morphogenesis.
A simple evolutionary model shows that lifetime learning can reshape the fitness landscape and accelerate discovery of inherited solutions. Acquired settings need not be copied into the genotype for learning to guide selection.
Characterizes neighborhood-aggregation graph networks through their ability to distinguish graph structures. It relates expressive power to Weisfeiler-Lehman tests and proposes an architecture with stronger discriminative aggregation.
Analyzes post-mortem human hippocampi across age, cognitive status, and Alzheimer's disease using molecular profiling. The study investigates neurogenic cell populations and regulatory networks, linking differences in neurogenesis-related states to the sampled conditions.
Uses fMRI representations of learned objects to examine hippocampal replay during quiet rest. Replay prioritizes weakly learned information and relates to subsequent memory performance, connecting offline activity with selective consolidation.
Introduces human niche construction as reciprocal interaction between organismal activity and selection environments. It emphasizes ecological inheritance and culturally transmitted knowledge alongside genetic inheritance.
Uses single-nucleus sequencing, proliferation markers, and computational identification to find neural progenitor stages in human hippocampus. Adult proliferating progenitors are sparse compared with childhood populations.
iLLaDA scales masked-diffusion language-model training with bidirectional attention and a consistent objective through supervised fine-tuning. Variable-length generation and evaluation changes accompany improved results over the earlier diffusion model.
IFEval evaluates instruction following through automatically checkable constraints such as word counts and required terms. It supplies prompts and reproducible checks to reduce reliance on costly human or potentially biased model judgments.
Measures iron-isotope compositions of micropyrites in ancient stromatolites. Spatially resolved geological evidence is used to investigate intense iron cycling and possible metabolic contributions while considering abiotic and alteration alternatives.
A discrete-time logistic rate-neuron model adapts intrinsic gain and bias to reduce output-distribution KL divergence from a bounded exponential target with specified mean. The derivation differentiates the transfer function for an input distribution, and uses a small fixed learning rate to track changing statistics rather than ensure exact convergence. A self-excitatory single-neuron example moves into a limiting cycle. Random recurrent networks with fixed signed synapses exhibit synchronized oscillatory, chaotic or intermittent bursting regimes depending on excitation fraction, connectivity and target mean. The authors acknowledge the two-parameter transfer family cannot generally fit the target distribution exactly. Their discussion proposes an interplay with synaptic plasticity; that combined system and an embodied environment are not demonstrated in the reviewed model.
Analyzes the relationship between logical irreversibility and physical heat generation. The paper argues that operations losing distinguishable information impose thermodynamic costs, separating erasure-related limits from an assumption of heat for every computational step.
Tests whether model collapse depends on replacing past training data. Language-model experiments confirm degradation under recursive replacement but find that retaining and accumulating real and synthetic data can avoid collapse in the studied settings.
Evaluates GPT-3 on tasks supplied through instructions and a few examples without task-specific gradient updates during evaluation. Scaling pretraining yields substantial in-context task performance while leaving uneven capabilities and limitations.
LLaDA uses masked diffusion rather than left-to-right factorization for language modeling. A Transformer predicts masked tokens during reverse generation, with pretraining and instruction-tuning evaluations comparing it to autoregressive baselines.
Tests intrinsic reasoning self-correction without external feedback. In the evaluated settings, revision often fails to improve answers and can degrade them, challenging assumptions that another self-critique pass reliably fixes errors.
Layer normalization centers and rescales a layer’s summed inputs using statistics across hidden units for each individual case, followed by per-unit gain and bias before activation. In recurrent networks the statistics are recomputed at each time step and learned gains/biases are shared across time. It supports a single-case online regime and has invariances different from weight or batch normalization: rescaling an entire layer is distinct from scaling one neuron’s incoming weights. The paper frames the method as improving gradient training and stabilizing hidden-state scale.
Documents methodological problems in language-model evaluation, including setup sensitivity, inconsistent comparisons, and weak reproducibility. Experience with the evaluation harness motivates concrete reporting and comparison practices.
Presents the original LLaMA model family trained on publicly available data. Architecture and training choices yield strong benchmark results at smaller sizes than some contemporary models.
LLM-Pruner removes coupled structural components using gradient-based importance and then recovers capability with limited additional training. It targets practical model compression while minimizing dependence on the original training corpus.
Constructs logically reversible simulations of general computations by retaining intermediate history, copying the output, and reversing unwanted work. This shows that useful computation need not require irreversible information loss at every step.
LoRA freezes a pretrained model and learns low-rank additions to selected weight matrices. Experiments demonstrate parameter-efficient task adaptation with much smaller trainable state than full fine-tuning.
Reports persistent recent-memory impairment following bilateral medial-temporal resections involving the hippocampal region. Formal testing across surgical cases connects lesion extent with memory outcomes.
Controlled question-answering and retrieval tasks show that relevant information is used unevenly across long contexts. Performance often falls when the needed material occurs in the middle rather than near the beginning or end.
Distinguishes forming a cooperative group from transforming it into an integrated evolutionary individual. The review analyzes roles of communication, dependence, division of labor, conflict, and ecological conditions in major transitions.
Mamba makes state-space updates input-dependent so sequence processing can select what to retain or forget. A hardware-aware recurrent implementation achieves linear sequence scaling and is evaluated across several modalities.
Introduces a broad multiple-choice benchmark spanning academic and professional subjects. Initial experiments expose uneven accuracy and poor awareness of errors even in large language models.
Describes intra-layer tensor parallelism for training large Transformers across GPUs. A small set of communication operations distributes matrix computations, with scaling experiments measuring throughput and efficiency.
Searches for synapse-specific Hebbian learning rules rather than fixed final weights. Randomly initialized networks change during execution and are evaluated on reinforcement-learning tasks, including high-dimensional sensory and locomotion settings.
Combines brain glucose-use estimates with body-growth data across development. The study finds high childhood brain metabolic demand and examines its relationship to slow body growth and human developmental evolution.
Presents a compact language model using grouped-query and sliding-window attention for efficient inference. Benchmark comparisons and an instruction-tuned variant illustrate capability relative to larger contemporary models.
Mixtral routes each token through a small subset of multiple feedforward experts. Sparse activation provides access to a larger parameter pool while limiting computation per token; base and instruction models are evaluated.
Averages weights of models fine-tuned from a common pretrained starting point. Experiments find improved accuracy and robustness without ensemble inference costs when the solutions occupy compatible low-error regions.
Analyzes seventy-two experimentally supported discrete models of cell processes and finds long-timescale order stronger than edge-of-chaos expectations. Perturbation propagation is often transient or has limited phenotypic effect.
Studies developmental interneuron survival and shows that activity can modulate apoptosis. Changes in interneuron populations and transient depolarization reveal feedback regulating cortical inhibitory-cell number and repertoire.
Audits language models trained against reference-free model judges and finds improved judged plausibility without corresponding correctness gains. Hidden external checks reveal reward hacking and transfer of plausible errors across judge models in the tested setting.
Establishes broad approximation results for feedforward networks with suitable nonlinear hidden units. The analysis treats network expressiveness as a property of function classes rather than of a specific training algorithm.
Uses spatially resolved sulfur-isotope measurements in ancient sulfate minerals to examine atmospheric chemistry and sulfur cycling. Different isotope populations and comparisons with modern aerosols complicate simple interpretations of ancient anomalies.
Carbon dating of neuronal DNA and labeling evidence indicate little or no detectable postnatal neuron generation in the sampled human neocortex. The paper contrasts developmental neuron production with adult persistence.
Net2Net expands neural networks through function-preserving widening and deepening transformations. Transferring an existing solution to a larger model accelerates subsequent training compared with starting from scratch.
Analyzes statistical and topological features of the adult fly whole-brain connectome. Spatial information, neuron classes, and cell labels connect wiring patterns and circuit motifs to potential organization of information flow.
Unifies graph-based molecular prediction architectures as message-passing neural networks. Learned messages, aggregation, and graph readout are compared on quantum-chemistry prediction tasks.
Genetic programming evolves expressions that combine parental weight vectors. Pair-specific disposable operators with a rising crossover rate improve final validation cross-entropy relative to no crossover on five classification datasets in 30-run comparisons. Simple mean crossover has significant benefits on only two datasets. Initial fixed-rate screening favors reusable operators, while the later schedule favors disposable ones; reusable operators converge to arithmetic means. Mutation uses gradient descent and pair-specific crossover adds inner search/evaluation work.
Studies how local inhibitory circuitry regulates quiescent adult hippocampal stem cells. Tonic GABA signaling and receptor manipulations alter exit from quiescence and self-renewal, linking circuit activity to progenitor decisions.
Reconstructs an adult fly brain's neuronal wiring from electron microscopy with extensive automated reconstruction and proofreading. The released connectome enables comprehensive circuit tracing and network analyses.
Experiments with coacervate droplets in a temperature-driven pore show accumulation, growth, fission, and condition-dependent selection. Convective and interfacial processes couple protocell-like behavior to nonequilibrium environmental conditions.
Pairs an odor with declarative learning and re-presents it during slow-wave sleep. Memory tests and imaging support selective reactivation and improved consolidation under the relevant sleep condition.
Describes the Olmo 3 family and releases its model-development flow, including stages, checkpoints, data, and dependencies. The report covers long-context reasoning, coding, tool use, and instruction-oriented post-training.
Introduces OLMo with model weights, training data, training code, and evaluation artifacts released together. The project aims to make language-model behavior scientifically inspectable beyond access to final weights alone.
Proves that finite recurrent networks with rational coefficients and suitable sigmoidal dynamics can simulate Turing machines. Explicit constructions and complexity analysis show computational power without higher-order multiplicative networks.
Proposes an operational definition of a self-sustaining chemical system that includes reaction networks, inflow, and initial seeds. Seed dependence links establishing the system to a proposed form of heredity.
Analyzes vanishing and exploding gradients in recurrent networks through geometric and dynamical arguments. Gradient-norm clipping and a soft constraint are proposed and evaluated as training remedies.
Reconsiders thermodynamic accounts of life, separating macroscopic order from microscopic entropy. It proposes viewing persistent organization as enabling metabolic channels through which entropy can increase.
Proves Turing-completeness results for specified Transformer and Neural GPU architectures. The constructions identify internal representations and architectural elements sufficient for general computation under the stated assumptions.
Investigates complexity measurements in Tierra and discusses why continuing artificial evolution often exhausts a limited environment. It distinguishes open-ended novelty from simple complexity growth.
Summarizes workshop perspectives on open-ended evolution and identifies multiple distinct forms. It separates observable hallmarks from proposed mechanisms and outlines unresolved research questions.
Dynamic mean-field analysis of driven random leaky-tanh networks distinguishes local instability from asymptotic conditional chaos. Replicas share the same weights and fluctuating input. A locally expansive but nonchaotic regime appears, and the network contribution to linear sequential memory peaks there. Stronger chaos can obscure input information. Input-driven activity and correlations persist even when initial-state differences decay.
Combines species-resolved metagenomics with chemical and spatial measurements in a hypersaline microbial mat. The study maps distributed resource capture and transformations across a compact structured community.
Compares molecular-clock predictions of early animal divergence with fossil and ecological evidence. It argues that major changes in organization should produce ecosystem effects that can help test timing hypotheses.
Elastic weight consolidation protects earlier tasks by slowing changes to parameters estimated to be important for them. Sequential-task experiments show improved retention compared with unrestricted learning.
Introduces counter-based pseudorandom generators, including Philox, that map keys and counters to samples without sequential mutable stream state. Statistical and performance evaluations target massively parallel computation.
GOMEA learns a family of variable subsets, commonly a mutual-information linkage tree, and copies donor values for a subset before keeping or reverting the change according to observed fitness. Subsets can be nested, overlapping and noncontiguous in storage. CGOM restricts donors using dependent variables outside the current subset. Forced improvement uses elitist material and can eventually copy an elite solution. In nine binary optimization benchmarks, CGOMEA-P3 improves on GOMEA-P3 on five, four significantly, but performs worse on HIFF. Evaluation efficiency does not uniformly imply faster runtime; population-size automation leaves other choices, including a tuned threshold.
Single-neuron recordings during human working-memory tasks identify persistent activity with different roles across medial temporal and frontal regions. Stimulus-specific and task/load-related patterns predict aspects of memory behavior.
Uses expanded genome sampling and phylogenomic analyses to investigate mitochondrial relationships, proposing a common ancestor with the SAR11 clade. The paper examines sensitivity to reconstruction methods and taxon choice.
GPO combines selected parents in behavior space: learn a state-dependent parent selector, then train a child by behavioral cloning with DAgger. Mutation uses policy gradients, optionally sharing samples. Nine MuJoCo tasks and two-task ablations support the combined method; data sharing supplies the largest isolated gain, with crossover contributing further. Omitting crossover in the ablation copies the stronger parent. The timestep accounting includes imitation rollouts.
FractalDB generates labeled synthetic images using iterated affine transformations and parameter-defined categories, then pretrains a conventional ResNet-50 before natural-image fine-tuning. It uses fractals as data, not network wiring. Table6 reports CIFAR-100 accuracy62.7% from scratch,75.7% after FractalDB-1k,77.3% after FractalDB-10k and84.6% after ImageNet-1k pretraining. FractalDB-10k reaches50.8% on Places365 against50.3% for ImageNet pretraining but does not win all tasks. Table9 compares roughly million-image generated datasets: CIFAR-10075.7% for FractalDB-1k against71.1% for Perlin-1296 and68.1% for Bezier-1024. Category labels, rendering, filling rate and augmentations matter; larger image size/dot count are not uniformly better. Synthetic pretraining incurs substantial generation and optimization cost.
Examines how surprising changes to learned events alter hippocampal representations and later episodic memory. The study connects prediction mismatch with abandoning prior representations and incorporating new information.
Characterizes excitatory callosal connections in developing rat cortex with slice electrophysiology. The work examines synaptic response properties and whether this tract provides a tractable model of excitatory cortical communication.
Separates architecture evolution from reward-modulated Hebb/Oja/BCM weight adaptation. Authors report gains over fixed-weight NEAT on three continuous-control tasks, while PPO/SAC remain stronger on Hopper and BipedalWalker under reported interaction budgets. A simplified regret bound assumes bounded projected parameters, a finite architecture pool, full-information Hedge selection and surrogate domination. Appendix C.9 reports similar outcomes with and without inherited adapted weights.
PPO optimizes a surrogate policy-gradient objective through repeated minibatch updates on sampled interactions. Benchmark experiments compare sample efficiency, implementation simplicity, and runtime with other reinforcement-learning methods.
Shows in mice that pyramidal-cell activity during a developmental window influences interneuron survival. Synaptic input predicts survival, with PTEN-related signaling implicated in the regulation of inhibitory-cell numbers.
Models pyramidal-cell computation as nonlinear integration within separate dendritic branches followed by combination at the cell body. The resulting two-layer abstraction better represents the modeled input-output behavior than treating the cell as one simple threshold unit.
Pythia releases a controlled family of language models across sizes with shared data ordering and many intermediate checkpoints. Case studies use this record to analyze memorization, data-frequency effects, and changes during training.
QLoRA fine-tunes low-rank adapters through a frozen quantized base model. Quantization formats, double quantization, and paged optimizer state reduce memory demands while experiments evaluate retained task performance.
Qwen3 describes dense and mixture-of-experts models with multilingual training and unified thinking and non-thinking modes. A configurable thinking budget exposes an inference latency-versus-performance tradeoff.
Feedback alignment uses a fixed random backward matrix to propagate output-error-dependent learning signals while forward weights adapt. The artificial-network demonstrations show that useful multilayer learning need not transport an exact current transpose of forward weights. This does not remove targets, local activity factors, suitable feedback or adaptation assumptions, and a useful direction need not be the exact gradient at each step.
Examines when input-driven reservoirs forget initial conditions. Counterexamples show why a weight-matrix spectral-radius heuristic is not a general nonlinear echo-state guarantee. Stronger stability conditions and properties of the driving input matter.
Recurrent Memory Transformer adds memory tokens passed between sequence segments. Training teaches these tokens to carry compact information, allowing segment recurrence without redesigning the core Transformer.
RLEdit treats sequential hypernetwork-based model editing as a reinforcement-learning problem. The update generator is trained to accommodate changing model parameters and retain earlier edits across knowledge sequences.
ReST alternates generating samples from a language policy and improving it using filtered data and offline reinforcement learning. Machine-translation experiments investigate efficient reuse of samples and established quality rewards.
Somatic current-clamp recordings in juvenile rat layer-5 cortical slices compare repeated flat current pulses with repeated filtered Gaussian-current waveforms. Flat pulses preserve approximate spike count while timing drifts across trials; fluctuating stimuli can repeatedly lock spikes with submillisecond precision. Figures 1–3 distinguish trial-to-trial count, event reliability, timing precision and spike-triggered stimulus averages. The best response block for each of ten cells is used for the high-reliability/precision comparison. Repeated fluctuations are the same waveform across trials, not independent new noise on every presentation. Synaptic receptors are blocked to isolate spike generation. Stimulus filtering, amplitude, small background noise and adaptation matter. The experiment does not establish precision at synaptic transmission or prove a timing code is physiologically used.
Comparative analysis of 227 PRDM9 orthologs from 225 vertebrate species finds repeated losses or partial-domain variants. Swordtail fish retaining partial PRDM9 show recombination enrichment near transcription start sites, CpG islands and H3K4me3 rather than predicted PRDM9-binding motifs. Ancestry-switch mapping used 268 sampled natural hybrids, of which 239 passed the read-count filter; median breakpoint resolution was 13 kb. The authors discuss possible costs of directing exchanges near functional elements, but these evolutionary explanations are hypotheses rather than experimentally established reasons for PRDM9 evolution.
Yeast defined-base-pair reversion assays combine proofreading-defective polymerases with mismatch-repair deficiency. Overlapping proofreading and mismatch-repair layers suppress errors, and polymerase δ can proofread errors in trans whereas ε chiefly acts in cis in these assays. Certain 8-oxoG-associated mismatches escape proofreading. Reporter orientation and local sequence context constrain the inference; this is not a genome-wide universal mutation-rate estimate.
Reviews bacterial chemotaxis, especially temporal sensing and run-and-tumble control in E. coli. Receptor signaling, adaptation, and biased movement explain how cells navigate gradients despite noisy local measurements.
RetNet combines parallel training, recurrent inference, and chunkwise recurrent processing through a retention mechanism. Language-model experiments evaluate this alternative to conventional Transformer attention.
Retrieval-augmented generation combines a pretrained sequence model with a searchable nonparametric document memory. Jointly used retrieval and generation improve evaluated knowledge-intensive tasks and can provide more specific responses.
RL-squared trains a recurrent network through a slower reinforcement-learning procedure so its runtime hidden state implements rapid adaptation. Observations, actions, rewards, and episode indicators are inputs to the learned algorithm.
RoFormer introduces rotary position embeddings that encode position through rotations of query and key representations. Their interaction carries relative-position dependence within attention, with evaluations of the resulting architecture.
RMSNorm removes mean-centering from layer normalization and rescales by root mean square. Experiments examine whether this simpler operation retains useful training behavior while reducing computational overhead.
RULER extends simple long-context retrieval tests with multiple targets, tracing, aggregation, and varied difficulty. Evaluated models often fall short of effective use across their advertised context windows.
RWKV combines time-based recurrent mixing with a formulation supporting parallel training. It aims for constant per-step inference state and computation while retaining language-model capability at scale.
The authors align hidden neurons using their activation responses to common data, with pairwise correlation or CCA-derived matching, then interpolate consistently reordered weights. Tests on 200 independently trained network pairs per dataset show less damage than unaligned interpolation. On CIFAR-10, midpoint accuracy is approximately 25% with alignment versus 10% naïve, below parents near 40%. These are fixed-architecture feedforward interpolation experiments, rather than whole evolutionary search comparisons.
Safe mutation scales perturbations using change in outputs on sampled parental experience. SM-R uses a line search along a random direction; SM-G scales weights by output sensitivity, with recurrent sensitivities propagated through time. These are output gradients rather than objective/reward gradients. The GA has no crossover, tournament size five, and separately tuned mutation variants. Results vary by task and variant: SM-R performs poorly on the difficult maze and SM-G-SUM provides no advantage over control there. Avoiding additional environment interactions still entails neural forward/backward computation.
Studies how relevant nodes and attractor behavior scale in ordered and critical random Boolean networks. Analytical arguments and numerical experiments show different growth patterns across dynamical regimes.
Fits empirical language-model loss trends across parameter count, data size, and training compute. The resulting scaling relations guide allocation of a fixed training budget within the studied regimes.
Compiles biodiversity data to investigate commonness, rarity, dominance, and richness scaling across microbes and larger organisms. Extrapolated scaling relationships yield very large estimates of global microbial diversity.
SEAL has a language model generate adaptation data and update directives, then applies supervised parameter updates. Reinforcement learning trains useful self-edits using downstream performance after adaptation.
Trains neural cellular automata so local pixel-like cells communicate and reach a distributed classification of digit shape. Demonstrations examine adaptation to drawing changes and robustness under selected distortions.
Self-Instruct bootstraps instructions and examples from a pretrained language model, filters invalid or redundant generations, and fine-tunes on the resulting data. Evaluations show improved instruction following.
The paper constructs feedforward self-normalizing networks using SELU activations and weight statistics so that a mean/variance map has an attracting fixed point within a stated domain. The mapping uses independent inputs sharing moments and an approximately Gaussian weighted sum motivated by the central limit theorem. It handles restricted deviations of weight sum and squared norm, and presents computer-assisted contraction reasoning whose detailed proof is in supplementary material. Its normalization target concerns activation moments across samples/layers.
Trains neural cellular automata against texture and feature targets, producing persistent evolving patterns. Analyses examine hidden states, robustness, and unexpected properties of the learned local rules.
SPIN iteratively contrasts model-generated responses with retained human demonstrations. Starting from a supervised model, self-play-based fine-tuning improves evaluated responses without collecting new annotations at each iteration.
Self-Refine uses the same language model to draft, critique, and revise outputs at inference time. Human and automatic evaluations across several tasks favor refined outputs without an additional parameter-training stage.
Self-Rewarding Language Models generate their own preference judgments and train through iterative DPO. Experiments report improved instruction following and judging quality across a small number of iterations.
SentencePiece trains subword tokenization directly from raw sentences without requiring language-specific word segmentation. C++ and Python implementations and translation experiments demonstrate the approach and compare configuration choices.
Attention-based set encoders model interactions between elements. SAB/ISAB are equivariant; pooling produces an invariant output. Inducing points give O(nm) attention rather than full O(n²). The max-regression table favors max pooling over the attention model, cautioning against blanket superiority.
Introduces a compact dynamical spiking-neuron model capable of diverse firing and bursting patterns. It seeks a compromise between conductance-model richness and the computational efficiency of simpler integrate-and-fire models.
Constructs self-reproducing universal arrays using cells with small internal programs. The design trades richer elementary automata for simpler programming than earlier constructions from very simple finite-state cells.
SmolLM2 develops a small language model through extensive multi-stage training and specialized mathematical, code, and instruction data. Ablations and data-mixture refinement explain how corpus quality and allocation affect capability.
SparseGPT prunes large pretrained Transformers in one pass using efficient reconstruction of remaining weights. Evaluations include unstructured and hardware-oriented sparsity patterns, with limited quality loss without retraining.
Evolves networks under goals that switch among combinations of reusable subgoals. Compared with fixed goals, these modularly varying environments promote modular architectures and recurrent motifs in the tested systems.
STaR generates reasoning traces, checks answers, and fine-tunes on successful rationales. Failed attempts can be re-generated with the correct answer supplied, enabling iterative improvement from limited initial rationale examples.
Analyzes elementary cellular automata as statistical systems with local deterministic rules. The study develops patterns, dynamical classifications, and statistical behavior as a foundation for investigating self-organization.
Derives thermodynamic constraints on self-replication in a system coupled to a thermal bath. A lower bound on heat production relates replication to irreversibility, internal entropy, and the possibility of decay or reverse processes.
Random continuous-time rate networks driven by periodic inputs with random neuronal phases can transition from chaotic background activity to entrained periodic motion. The critical drive amplitude depends nonmonotonically on frequency. Mean-field calculations are compared with network simulations; suppression can occur without simply saturating the entire network. Connectivity alone therefore does not fix the activity regime independently of input.
Strongly typed genetic programming constrains tree construction and mutation by required return/argument types. Crossover chooses a node in the first parent and compatible-return-type candidates in the second; if no legal match exists it produces no crossover offspring or returns parents. Instantiated generic types must remain consistent after inheritance. These rules avoid many syntactically or type-invalid programs; the selected reading does not establish a quantified performance advantage.
Reviews structural features of genotype-phenotype maps, including redundancy, robustness, neutral networks, and biased accessibility of phenotypes. Different model systems illustrate how the map constrains evolutionary exploration.
Surrogate tests compare a declared null hypothesis with data, rather than establish chaos whenever traces differ. Nonlinear observation of a linear stochastic process can create bursts. AAFT can bias spectra; iterative refinement improves constraint matching but requires residual checks. Nonstationarity and external kicks can cause null rejection. Nonlinearity testing does not establish determinism, a finite attractor dimension or chaos.
Switch Transformers simplify sparse expert routing and address communication and training instability. Selected experts increase parameter capacity without activating the entire model for each input, with large-scale training demonstrations.
Paired stimulation of cultured hippocampal neurons produces potentiation or depression according to relative spike timing. Initial synaptic strength and postsynaptic cell type also influence the changes.
Anoxic cultured hippocampal neurons are protected by blocking excitatory amino-acid signaling. Parallel physiological tests support transmitter-mediated depolarization as a causal contributor to neuronal death under oxygen deprivation.
Volume electron microscopy and three-dimensional reconstructions reveal membrane continuity in a ctenophore nerve net, forming a syncytium. This architecture differs from a simple assumption that all nervous systems consist only of separate connected cells.
Builds biophysically detailed neuron models from recorded morphology and electrophysiology. Genetic optimization fits conductance and other parameters, and additional stimuli evaluate how well the models reproduce responses.
Maximal Update Parametrization supports transferring tuned hyperparameters from smaller neural networks to larger ones. Theory and experiments examine settings in which useful learning-rate and related choices remain stable with scale.
Studies test-time adaptation tied to next-token prediction in pretrained language models. Fast-weight updates use observed causal information to target contextual states, with long-context evaluations across several model families.
Tests a genomic-bottleneck hypothesis by limiting the repertoire of Hebbian learning rules and learning their synapse assignments. Joint optimization performs poorly in the studied environments, contrary to the initial expectation of improved generalization.
Identifies synaptotagmin 7 as a calcium sensor required for synaptic facilitation at multiple tested synapses. Repeated presynaptic activity can increase subsequent transmitter release through a mechanism distinct from immediate release sensing.
Compares direct, static-generative, and temporal-developmental robot encodings under matched parameter and evaluation budgets. Development increases phenotypic diversity but reduces mutation locality and recombination robustness without reliably improving final fitness.
Analyzes a ctenophore genome and neural molecular machinery, finding substantial differences from other animal lineages. The paper uses these observations to investigate competing accounts of nervous-system evolution.
Mouse mutant and explanted-ovary irradiation experiments support CHK2-dependent elimination of many oocytes carrying unrepaired DNA breaks, including severely asynaptic SPO11-deficient oocytes. HORMAD2 loss accelerates disappearance of repair foci and alters the barrier to sister-mediated repair. CHK2 deletion does not fully restore mutant oocyte numbers: quality control involves additional mechanisms. The reported break threshold is inferred from irradiation and RAD51 foci in a particular developmental context, not a universal safety threshold.
Studies mutation and recombination in a linear genetic-programming system using genotypic and phenotypic measures. Recombination promotes exploration and robustness and is less phenotypically disruptive than mutation in this setup.
Tracks complete genealogies of evolving digital organisms as complex logic functions arise. Useful simpler functions provide stepping stones, and mutations can change their effects as the genomic context develops.
Documents FineWeb corpus construction from web snapshots and tests deduplication and filtering choices. An educational subset improves selected knowledge and reasoning benchmarks in the evaluated training conditions.
Characterizes the genotype-phenotype map of digital organisms using computed logic functions as phenotypes. The analysis finds uneven phenotype abundance, neutral connectivity, and asymmetric transitions related to robustness.
Describes the Llama 3 family, including multilingual, coding, reasoning, and tool-use capabilities. The report covers large-scale pretraining, post-training, empirical evaluation, and exploratory integration of other modalities.
MODES proposes measurements of change, novelty, complexity, and ecological potential in evolving systems. Detailed algorithms and example implementations aim to support comparisons of observable open-ended-dynamics hallmarks.
Proposes a simulated peptide/RNA-world pathway to translation and genetic-code organization. Encapsulation and molecular interactions are modeled as enabling successive components of a prebiotic information system.
Investigates undirected bacterial motility in physically structured, resource-limited habitats. Experimental and modeling arguments support an advantage from moving away from neighbors and accessing unsaturated regions even without directional chemotaxis.
Reviews Purkinje-cell spine structure, formation, stabilization, and interactions with parallel and climbing fibers. It distinguishes mechanisms of protrusion from contact-dependent maintenance and examines uncertain estimates of spine abundance.
Temporarily updates language-model parameters at inference using losses derived from available task examples. Experiments on abstract reasoning and few-shot tasks show substantial gains under the proposed test-time-training procedure.
Derives limits on fixed-depth self-attention for representing selected periodic and hierarchical formal languages as sequence length grows. The analysis distinguishes mathematical expressiveness under resource constraints from practical approximate language performance.
An editorial thread introduces differentiable self-organizing systems and brings together neural cellular-automaton demonstrations and expert commentary. It asks how simple local learning rules can produce robust collective behavior.
TIES-Merging combines fine-tuned models by trimming small parameter changes, resolving sign conflicts, and aggregating aligned updates. Experiments show reduced interference compared with simpler model-merging approaches.
TinyLlama trains a compact language model using Llama-family architecture and tokenization with efficient implementation choices. Released code and checkpoints support experiments on smaller models despite extensive pretraining.
Titans adds a neural long-term memory that updates at test time alongside shorter-term context processing. Surprise-sensitive memory updates and architectural variants are evaluated for sequence tasks and long-context use.
A computational network uses electrotonically segregated feedforward and feedback compartments and differences between ordinary and target-driven phases to obtain local plateau-related teaching signals. Fixed random feedback, phase coordination and settling/averaging remain assumptions. Connections between local hidden objectives and output improvement have explicit conditions, including a small output discrepancy and a Jacobian condition; extra depth did not provide unrestricted improvement.
Uses a large sweep of language-model sizes and token budgets to estimate compute-optimal training allocation. Results favor scaling training data alongside parameters and demonstrate a smaller, more thoroughly trained model under a matched compute budget.
Reduces neural-training memory by storing selected activations and recomputing others during backpropagation. Graph analysis and checkpoint schedules expose a controlled tradeoff between memory and extra computation.
InstructGPT combines demonstration-based fine-tuning, learned human-preference rewards, and reinforcement learning. Human evaluations show improved alignment with user intent compared with larger untreated language models in the tested setting.
SCoRe trains multi-turn self-correction through online reinforcement learning. The method targets failure modes of naive correction training and evaluates improvements in revising reasoning answers without requiring external feedback at each inference revision.
Introduces GSM8K mathematical word problems and trains verifiers to rank generated solutions. Sampling several candidates and selecting by a learned verifier improves evaluated performance over direct fine-tuning baselines.
Transformer-XL carries prior-segment representations through segment-level recurrence and uses a compatible relative positional scheme. Evaluations show better long-range language modeling and reduced repeated computation during inference.
Develops state-space duality linking selective state-space models and attention variants through structured matrices. This framework motivates Mamba-2 and a faster core implementation with language-model evaluations.
Estimates ecosystem entropy production from flux and meteorological data at Amazon forest and pasture sites. The comparison finds higher rates in forest and discusses whether this relates to ecosystem development and maximum-production hypotheses.
Shows that standard pair-based spike-timing plasticity cannot reproduce several experimental protocols. A triplet-based rule with multiple decaying event traces better captures frequency and higher-order timing dependence.
Tulu 3 releases a full post-training recipe combining supervised fine-tuning, preference optimization, and reinforcement learning with verifiable rewards. Data, evaluation, code, and intermediate artifacts support inspection of behavioral changes.
Constructs a finite recurrent sigmoidal neural network capable of simulating a universal Turing machine without higher-order connections. The proof intentionally separates computability from training and practical implementation.
Studies initialization and nonlinearities in deep supervised feedforward networks. The initial input-count uniform heuristic differs from the proposed fan-in/fan-out compromise. Variance reasoning assumes approximately linear activations and independence; logistic saturation and depth affect training. Classification experiments support the proposed scheme in those settings.
Uses PET measurements of oxygen extraction and glucose utilization to test regional metabolic differences in healthy resting brains. Results support largely uniform oxidation-related measures across the examined resting-state networks.
Universal Transformers repeatedly apply a shared self-attention transformation across depth, with an option for adaptive computation. Experiments address limitations of fixed-depth sequence models and test algorithmic generalization.
Adds rapidly changing weights as a temporary associative memory between fast neural activity and slowly learned parameters. Recent hidden states influence subsequent computation, providing attention-like access to the recent past.
Mice learned an experimenter-defined neurofeedback task using two intermingled retrosplenial layer-5 neuron populations with opposing decoder roles. Relative distal-versus-proximal calcium event magnitudes carried task information and opposing role-dependent signs. Surrounding-network activity predicted part of those residual signals. Perturbing dendritic processing through NDNF-positive layer-1 inhibition impaired learning, with light controls and decoder recalibration to address activity suppression.
Weight normalization parameterizes a weight vector by its direction and a separate scale, w=g v/||v||, and trains those parameters by stochastic gradient descent. It alters gradient scaling/projection and conditioning, with a monotonic norm argument specifically for ordinary gradient descent without momentum. It is equivalent to batch normalization only under restrictive whitened-input/single-layer conditions. The paper supplies a data-dependent one-minibatch initialization for feedforward settings, explicitly noting recursion requires standard initialization instead.
Reports acid-catalyzed nucleotide polymerization during wet-dry cycles involving thin water films or droplets. Experiments investigate an environmental route from monomers to longer chains relevant to prebiotic chemistry.
The preserved chapter-six excerpt of Schrödinger's historical book examines order, disorder, and entropy. It asks how organisms avoid equilibrium through metabolism and environmental exchanges, and clarifies the famous negative-entropy formulation through a discussion of free energy.
Compares NEAT, HyperNEAT, CMA-ES, a simple MAP-Elites variant, PPO and goal-conditioned PPO in logic-circuit curricula and robotic tasks. Direct encodings perform better in the tested transfer curricula, while HyperNEAT can escape a simple deceptive maze. Larger robotic morphology removes several apparent advantages. Diversity coverage alone need not retain the ordered behavior a changed task requires.
YaRN extends the usable context of rotary-position language models through positional interpolation and associated adjustments. Experiments evaluate longer-context adaptation with less training than earlier extension methods.
ZeRO shards optimizer state, gradients, and parameters to remove redundant memory in distributed training. Memory and communication analyses plus large-model experiments show how additional devices can support larger models.