Pack the day bag for the transitions, not for the weather. A Tokyo autumn day runs cool before nine, warm by early afternoon, cool again after dark, and can drop a shower into the middle of it with no useful warning, so the bag needs a removable layer, a compact umbrella with a waterproof pouch for your phone, a portable charger, and a small hand towel, worn crossbody or as a small backpack so both hands stay free. Everything past that list has to earn its place.

I learned the shape of this by walking out of a station on what felt like a jacket morning and then standing on a platform at two in the afternoon holding that jacket, with nowhere to put it, because the bag on my shoulder was a flat thing that held a phone and a wallet and nothing else. The jacket stayed over my arm for six hours. That is the failure the day bag exists to prevent, and it isn't cold or rain. It's the slow accumulation of things you can no longer put down.
Size the bag for six in the evening
In the morning the bag should look slightly too empty. If it's full when you leave, then by mid-afternoon you're carrying a shed layer, a damp umbrella, a wrapper, and whatever you picked up, and it has turned into a weight pulling on one shoulder all day. I pack for the load I'll have at six, and I put up with it sliding around at eight.
That has a second effect people don't expect: it stops you from treating the bag as storage. A day bag in Tokyo is a working tool, not a suitcase you happen to be wearing. Anything you won't touch before dinner is a candidate for the hotel room.
The layer, and where it goes when you take it off
Autumn here isn't one temperature. The morning is cool enough that you want something over a shirt, midday is often warm enough that you don't, and by the time you're walking back after dark you want it again. So the layer has to be the kind you can take off and put away fast. A light jacket or a cardigan, not something that needs a decision.
Then it has to have somewhere to live, and that's the part that gets skipped. A layer you can't stow is a layer you carry in your hand, which means one hand is gone for the rest of the day. If the bag has a flat outside pocket or a bit of slack in the main compartment, the jacket goes in folded and comes back out at six.
I keep a scarf in there too, through the shoulder seasons. It weighs almost nothing, it covers the gap between a shirt and a jacket, and it does double duty on a train car that runs warmer than the street.
Rain that starts while you're inside
Autumn rain in Tokyo often arrives without much theater. You're in a building, you come out, the pavement is dark and the drops are already steady.
A small folding umbrella that lives permanently in an outside pocket solves almost all of it, because the decision to bring it never has to be made. A cheap clear vinyl umbrella from a convenience store also works and is honestly better in a real downpour, but it's long, it's awkward on a crowded car, and you end up accumulating them.
The phone is the part to protect. Rain here is often light enough that you don't think about it until the screen is spotted, and a phone that lives in a wallet case with your transit card inside is also your fare gate, your map, and your translation. A sealable plastic pouch costs almost nothing, keeps it dry, and still lets you pull it out one-handed. The umbrella goes in the outside pocket wet. The phone never goes in the same pocket as the umbrella.
The charger is your train ticket
Most visitors now ride with a Suica or Pasmo loaded into the phone's wallet rather than a plastic card. That changes what a dead battery means. It isn't a missed photo. It's a fare gate that won't open, and a station agent who has to sort you out while the platform empties behind you.
The screen is what drains the battery, not the cold. Maps, train transfers, a camera app, a translation app, running on and off for ten hours straight. A small power bank and a short cable weigh very little and cover the whole problem. Some limited-express and shinkansen seats have outlets, but you can't plan a suburban line around that, and sitting in a cafe to charge means buying something and giving up a seat you may not get back.
Charge at the hotel and carry the bank anyway. The days you need it are the days with the longest walk home.
Hand towel, trash pouch, and the things nobody lists
Public restrooms in Japan often have no paper towels, and the dryers are sometimes weak or missing entirely, especially in parks and around shrine grounds. A small cotton towel about the size of your hand solves it, and it also handles a wet bench, a rain-spotted lens, or a spilled drink. It lives in the outside pocket with the umbrella, where you can reach it without opening the main bag at all.
The trash pouch is the other one people don't think about until they're holding an empty wrapper for an hour. Public bins in Tokyo are genuinely scarce, and the bins at convenience stores aren't always meant for whatever you're carrying. A small sealable pouch or a plastic bag keeps the day's odds and ends out of your pockets, and you empty it back at the hotel. Flat, it takes up no space.
A flattened tote stays at the bottom for two jobs: holding purchases so they don't mix with the wet umbrella, and holding the shed layer when the main compartment is already full. That's the real reason to carry a second bag. Separation, not extra volume.
Shape matters more than capacity
A crossbody bag sits at your hip and doesn't touch anyone else. During foliage season, when the ginkgo and maple run roughly from mid-November into early December, the cars serving the popular spots get dense and the platforms behind them get denser. On a car like that, a bag that stays against your body and leaves both hands free is the difference between moving through the crowd and being the obstacle inside it.
A small backpack is fine, but on a packed train it should come off your back or move to your front. Leaving it on is the one thing that will earn you a glare. And if the bag has a single opening, you'll be opening it on a moving train with one hand while the other holds a strap. Two pockets, one for the towel and umbrella and one for everything else, is the minimum that actually works.
You rarely need to carry water. Vending machines and convenience stores are on nearly every block, so a large bottle is weight you paid for and then carried all day. A small bottle you refill is enough, or buy as you go if you'd rather not think about it.
And if the day turns out genuinely warm, the layer is dead weight. Folded, it's a few ounces. I have never once regretted having it. That's the trade I'll take.