Jul 4, 2026

A broad survey of research at UPenn: my learnings

I have been at Penn for five semesters. In my sixth, I decided it was time to double-click on research.

Research is a strange thing to fund. The payoff is long-horizon, so it rarely makes sense for a private company to pay for it directly; it has to be subsidized by the government. And it is expensive. There is a lot of capex, and there are real economies of scale, so the talent and the resources centralize around a handful of institutions. If you spend all your time in the Wharton ecosystem, you can miss this completely. But it is there: huge facilities, and an enormous amount of effort poured into building and maintaining the machine that produces new knowledge. In America it is easy to assume the companies do all the work; often they are just sitting on top of intellectual accomplishments that came out of universities.

Something has shifted recently, though. A lot of researchers are leaving for venture-backed companies. OpenAI and Anthropic showed that you can do real research in a space that also commercializes, and that the economics can actually work. Robotics is one of those spaces. But this is not true of every field, and it has not been true for all of time.

So this semester I sat in on a lot of talks, mostly GRASP seminars, plus some econ and theory ones. I took notes the whole way through, and here is the cleaned-up version, grouped by theme.

These rooms are small; twenty or thirty people on average, with only a handful of undergrads. I went to talks across a bunch of departments: robotics, machine learning theory, physics, neuroscience, and economics. Moving between them teaches you something. Each field asks its own kinds of questions; each one works at a different level of abstraction; and each one asks for a different kind of technical depth. You feel that difference once you are in the room.

You also get a sense of how fast each field moves. Some talks lean on ideas that are decades old; others point to things that happened last week. The robotics talks were almost always the second kind. That alone tells you a lot about where a field is in its life.

The most valuable part is seeing the people up close. You get a firsthand sense of where the smartest people are working and which problems they have chosen; you hear their backgrounds and the paths that got them there. After enough of these, you start to pattern-match what a strong researcher looks like.

The best talks also had something in common. The speaker would lay out how the field had changed over the last decade, and then place their own research inside that story. The work was not framed as one more incremental result; it was framed as a way to rethink what had been going on the whole time. The strongest researchers were the ones who could tell that kind of story. And a lot of being a researcher really is capital allocation. You get some funding from your institution, you have a fixed amount of bandwidth, and you have to accrue resources and then decide where to point them. The best program directors were the ones who made those moves well.

Alongside these seminars, I probably had ~30 one-on-one chats with PhD students, master’s students, and (many) undergrads who are deep into research. Those conversations were usually more personal.

I did keep an eye out for non-STEM fields (history, political theory, literature, arts), but just reading the titles of some talks immediately turned me off. It felt too woke¹. This is probably related to it being difficult to reality-test claims in the humanities. I’m sure there are great insights to be found, but it requires a finely curated filter, which I don’t have.

The talks are grouped by theme, and within each theme they run in chronological order.

Robotics & embodied AI

Jiafei Duan (Ai2 / MolmoAct, Feb 4)

Making a robot reason through visible intermediate steps beats an opaque end-to-end model, because you can actually inspect and fix where it goes wrong.

• The core move is reasoning in a continuous token space instead of discrete action tokens, with intermediate reasoning phases rather than a single end-to-end mapping
  • this shifts the model from a binary success/failure signal to something closer to chain-of-thought reasoning
  • the payoff is interpretability and diagnosis; you can see the intermediate steps and fix where it goes wrong, instead of debugging a black box

Chris Paxton (Agility Robotics, Feb 11), the one I keep coming back to

Robotics is fundamentally bottlenecked by data, and how cheaply you can collect good demonstrations matters more than any clever model.

• The whole premise: robot policies train on ~5 orders of magnitude less data than a model like Qwen, so how cheaply you can collect good demonstrations is the real game
• SOTA today leans on sim data, and human dexterity is hard to capture; we manipulate with the whole body (you fall backwards to open a fridge), not just a gripper
• Data-collection methods, by throughput (trajectories / 15 min):
  • bare human hands: 51, fastest but hard to transfer onto a robot
  • handheld rig (UMI): 35
  • teleoperating a real robot: 11, the worst on every axis: slow, expensive, and the human-robot latency becomes its own deployment problem
• Egocentric video is cheap but loses the dexterity signal; better tools like DexWild and UMI help; sim is cheap but mid
• World models are the bright spot: they generalize surprisingly well, even trained off unrelated CSGO recordings, though depth and spatial reasoning are still underexplored
  • an IDM pulls action tokens from video tokens, so you don’t need action-labelled data
• Two things plague deployment: the models are slow, and teleop has inherent human-robot latency
• Evals are bad and need standardizing
• Open threads: deployment as a data flywheel? what can robotics learn from self-driving? and humans use more than vision; touch/tactile data is basically unused today

George Konidaris (Brown, Feb 13)

Intelligence is embodied, and the real problem isn’t scale but compression; a robot has to throw away most of what it sees to build a useful representation.

• Opening provocation: nature is made of robots, not programs; intelligence is embodied
• Pushback on the current mood of throwing data at a transformer and expecting results: the older methods have been useful, don’t abandon them
• His unifying frame is “structuralism”: describe every subfield through one shared object, a decision process. The robot gets an observation and a reward. The reward is built from behavior preferences. It then returns an action.
• The deeper problem is compression. A robot has to throw away excess data and build a useful, abstracted representation. This holds for both observations and actions. That’s what shrinks the search space.
• Ended on the honest hard part: pixel-to-motor control is still brutal, and nobody really knows how humans build their own sensorimotor substrate

David Held (CMU, Feb 20)

You get both generality and precision not by cramming everything into one model, but by splitting scene reasoning from low-level motion in a hierarchy.

• The central tension: foundation models (VLA) are general but not precise; specialist models are precise but don’t generalize
• His answer is a motion hierarchy: a high level does scene reasoning, a low level handles how to move
• Sim is diverse enough to randomize objects, positions, and camera angles; good for an end-to-end policy, but it doesn’t generalize
• Hierarchical imitation learning bridges the gap: get a high-level representation first (a weighted point cloud), feed it to a low-level transformer-diffusion policy at every timestep
  • better generalization, and the weights are interpretable
  • he calls the system SMITH (Sim2Real Multi-Task Hierarchical)
• The counterintuitive result: naively grouping many skills into one model backfires; you get negative transfer across skills instead

Stone Tao (UCSD / ManiSkill3, Feb 25)

Fast-enough GPU simulation can replace slow, expensive teleoperation, and the simulation doesn’t even have to be realistic, just capture the few properties that matter.

• Thesis: raw simulation speed is what makes simulation-integrated learning viable; GPU sim + RL gives a strong policy fast enough that you don’t need slow teleop
  • this is what makes something like a $100 robotic arm realistic
• The headline demo, sim-to-real reactive cable routing, works despite unrealistic sim:
  • the simulated cables are just a bunch of tubes connected by joints
  • it transfers anyway because you only need two properties right: elasticity (returns to rest) and plasticity (the rest point itself drifts over time)
• Simulation-in-the-loop deployment: take an action step on a simulated copy of the robot, then motion-track it. You get collision avoidance for free, straight from the sim’s constraints.
• One honest limit is the credit-assignment problem. RL’s horizon is just too short.
• He’s also bearish on world models. The context is poor, they’re inconsistent, and they’re too expensive to run. They might help with high-level planning, but not as the controller.
• The tooling behind it all: ManiSkill3, a GPU-parallelized sim framework optimized for RL

Machine learning theory

Yuxin Chen (Penn / Wharton, Feb 19)

An LLM learning from a few prompt examples is secretly running an optimization algorithm inside its forward pass; no weight updates required.

• The puzzle: in-context learning; LLMs get better at a task just from a few prompt examples, with no gradient update
• The claim: a transformer implements an optimization algorithm in its forward pass. A linear transformer runs gradient descent across its layers. This works well for linear regression, less so for non-convex problems.
• A single pretrained transformer holds a “bag of functions”; in principle you can construct one whose prediction error → 0 on any given task
• The punchline (via Andrew Barron): any function can be linearly represented over n features, so in the limit the transformer is just running Lasso regression

Daniel D. Lee (Penn, GRASP director, Feb 27)

What a neural net really does is warp the geometry of data until a simple linear boundary can separate it, and studying that geometry is how you open the black box.

• Framing worry: we’re running out of usable training data; everything’s basically been used up
• Which is why double descent matters: past a complexity threshold, making the model even more complex actually reduces variance error again
• Core claim: what a neural net can do comes down to linear separability. A perceptron is just a hyperplane. So the real job of data augmentation is geometric: it warps a class’s manifold until a simple linear classifier can separate it
  • characterize each manifold by two numbers, radius and dimension, and ask how both change layer by layer
  • moment-of-covariance estimates (features × stimuli, from biological-neuron recordings) approximate the spectrum, but they’re biased. So study dimension collapse on small, unbiased models instead. There you can actually run the experiments.
• Closing advice (which I heard from nearly everyone senior): work on what you find interesting, because you can’t predict what will matter 10-15 years out

Daniil Dmitriev (Penn, Feb 27)

When an adversary can replay your old mistakes or watch your guesses, naive learning strategies break, and cleverer ones can still recover strong guarantees.

• Replay-adversary online learning: the learner outputs a hypothesis, nature reveals ground truth, but that truth might be a response to an old, replayed guess, not the current one
  • the learner must avoid the trap region where it can’t tell which is happening
  • take the simple threshold game. Guessing all 0s and updating on feedback is the naive move. A strategy that flips beats it. And against a merely stochastic adversary, you get a logarithmic bound.
• Private online learning: guess nature’s hidden bit, with an option to skip a round; the naive strategy already makes at most one mistake
  • the real question is differential privacy: can an outside observer infer the bit just from your sequence of guesses?
  • two defenses: an “above-threshold” mechanism (logarithmic), or a cruder “name and shame”

Physics & neuroscience

Roni Sengupta (UNC, Feb 6)

Reconstructing 3D from inside the body defeats foundation models because the physics of light on wet tissue is nothing like the photos they trained on; you need real geometric priors.

• The problem: turn a camera stuck in your lungs or GI tract into an accurate 3D model; huge clinical demand, still largely unsolved
• Why foundation models fail here: no comparable training data, and they can’t handle dynamic lighting or reflections; you need strong geometric features and new techniques, not just scale
• The key physical insight: color isn’t static: the near-field light effect means a surface’s apparent color depends on the direction and proximity of the light
• Directions she floated:
  • lighting scribble: diffusion models and encoders to relight
  • VFX-studio tricks: dust, splash, foreground/background interaction, aging, to synthesize data
  • maybe LiDAR to get ground truth for the tricky physics

Rylie Green (Imperial College, Feb 19)

Neural implants have stalled for 70 years because metal electrodes are the wrong material; soft, ion-friendly hydrogels that neurons can integrate into are the way forward.

• The provocation: the same core tech for 70 years (pacemakers, cochlear implants, all platinum electrodes), even though we’ve built newer things on top (bionic eyes, stentrode, Neuralink)
• Two limitations:
  • metals talk in electrons, cells talk in ions, so electrodes are inherently limited (like in the eye)
  • metals are stiff, nerve tissue is soft
• The fix is new electrode materials with high safe charge-injection limits and genuine softness
  • the right hydrogels, polyvinyl alcohol (reliable) plus gelatin, let neurons modify and integrate into the material rather than just sit next to it

Robert Jack (Cambridge, Feb 20)

Matter made of particles that burn energy to move produces collective behavior that ordinary equilibrium physics simply can’t describe.

• Active matter = systems whose individual particles consume energy to do work, instead of responding passively like ordinary matter
• The core observation is about mixing: these simulations show behavior that doesn’t appear in ordinary equilibrium statistical mechanics

Reza Sharif-Naeini (McGill, Feb 25)

Chronic pain from harmless touch traces to a specific set of inhibitory neurons failing in the spinal cord, which points to a drug target instead of an implanted battery.

• Touch-evoked pain arises one of two ways: (1) miswiring in the spinal cord, or (2) a lowered threshold for pain-sensing nerves (as in arthritis)
  • (1) is the dorsal horn mechanism (dorsal horn = part of the spinal cord), central to chronic pain, whose circuits are still poorly understood
• The clinical hook: a patient with 2 years of chronic thigh pain, treated with spinal-cord stimulation via a battery pack
  • his driving question: can you activate the inhibitory neurons pharmacologically instead of electrically?
• The mechanism: pain is relayed at the spinal cord. Inhibitory neurons normally suppress it through feedback. Nerve injury damages those inhibitors. The result is allodynia, pain from harmless sensations.
• The suspect is parvalbumin (PV):
  • remove PV neurons from healthy mice and they get hypersensitive to mechanical stimulus
  • inject the PV protein into PV-deficient mice and you get the same hypersensitivity
• Open questions he posed: can you overexpress PV? PV regulates calcium; can you target calcium directly? can you build a model of PV neurons? what supplies the inhibitory input onto PV?
• The real bottleneck: the specialized mice take ~2 years to breed; that’s the rate limiter on everything

Game theory & economics

Doron Ravid (UChicago, Feb 3)

How you model rational players, the strict Nash sense vs. the looser rationalizable one, changes your predictions most once games get networked.

• Rationalizability versus Nash equilibrium in games, and how that distinction plays out once you move into network games
  • I was completely lost for most of it; flagging so I actually go back and learn the difference

Ariel Rubinstein (Tel Aviv / NYU, Feb 17)

You can design an information environment where honesty is structurally forced, a cheater gives themselves away no matter what, without needing anyone to be strategic.

• The setup: a principal must award a prize to one of two parties, and only the parties themselves know who actually deserves it
  • neither party is strategic; each just follows a fixed coding procedure
• The trick: the principal writes a pair of texts such that, whatever procedure each side follows, a cheater becomes detectable, the one-way cheating principle
• King Solomon is the classic illustration: the setup exposes the liar without ever needing to be carried out

Hunt Allcott (Stanford, Feb 26)

Current EV subsidies mostly reward rich buyers who’d have bought anyway; the efficient and equitable fix is a new-car subsidy capped by price and income, and no used-car subsidy.

• The paper: efficiency versus equity of new vs. used EV subsidies
• The core finding: the new EV subsidy is regressive (it benefits the rich), so pair it with price and income caps, and add a separate used-EV subsidy
  • the data made it vivid: about 40% of transactions run through dealerships, with crazy bunching right at the $25K cutoff after the price cap
• The externalities net out positive: EVs carry greater manufacturing and weight externalities (more death in a crash), but pollution reduction still makes them better overall
• A subtle equity wrinkle: subsidizing used EVs raises used prices, which quietly benefits the rich sellers of new vehicles
• His bottom line, in terms of the marginal value of public funds: a new subsidy plus a price cap plus an income limit, and no used subsidy. It’s a Pareto call that values poor consumers over rich ones.