Arash @incrementaliser

Computational Linguist (PhD) 📖 | Interested in cognitively-plausible models of dialogue understanding 🧠🤖💬 | On the research scientist job market 🔦 incrementaliser.github.io Edinburgh, Scotland Joined February 2021

Tweets

309
Followers

123
Following

470
Likes

4K

Arash @incrementaliser

3 days ago

And that's also why I don't understand the compute+data+parameter scaling folks (aka bitter lesson fans).

0 0 0 8 0

View Details

Arash @incrementaliser

3 days ago

I know there's also garbage code going in, potentially the reason why models sometimes produce garbage. But then it's weird we are not caring enough about data efficient models that we can then feed controlled, higher quality data to.

1 0 0 16 0

View Details

Arash @incrementaliser

3 weeks ago

Shouldn't all these "skills" be already in the weights of the core model? I'm sure they are, but why is the model/agent not using them and users have to remind it of what it can do in the prompt?

1 0 1 86 0

View Details

Arash @incrementaliser

5 days ago

Some good amount of focus should go to coming up with better academic publishing (and preventing such misuse of AI) these days, rather than just publishing.

Delip Rao e/σ @deliprao

6 days ago

Okay, this is crazy. TMLR editors took the unusual step of inviting authors of a subset of papers before desk rejection to have a Zoom call and answer questions about their papers. Result: only 7 of 10 invited accepted, and 7 out of 7 failed to answer any questions. People are

19 67 562 79K 109

0 0 1 49 0

View Details

Arash @incrementaliser

a week ago

@sestesch This one is under my homepage repo which is private, but super easy if you ask your coding agent :)

0 0 1 38 0

View Details

Arash @incrementaliser

a week ago

Enter the embedding room! 🧙‍♂️ I was refreshing my memory on some NLP (it had to be the lectures of @chrmanning ) and realised all the visualisations of old embedding models that I’ve seen are 2D, and thought it would be cool to have a 3D “room” where you can fly through!

6 8 11 4K 9

View Details

Arash @incrementaliser

a week ago

@cursor_ai - Search + fly to a word - Phone view with a joystick Enjoy it live through the link attached! And let me know if you spot any bugs, or have ideas for any improvements. incrementaliser.github.io/blogs/embeddin…

0 0 1 66 1

View Details

Arash @incrementaliser

a week ago

the words! So I immediately asked @cursor_ai to build it for me.🙂 Features: - 10k words in a fly-through 3D room - Cosine UMAP for dimensionality reduction (300D → 3D) - Colours from k-means (20 clusters) - Supported models: word2vec and GloVe - Speed control

1 0 2 90 0

View Details

Graham Neubig @gneubig

3 weeks ago

Wow, can't wait to try out the new Fabl... Gemin... Muse Spark today!

4 5 101 15K 1

View Details

Tom McCoy @RTomMcCoy

3 weeks ago

🤖🧠NEW PAPER🧠🤖 (The result of an 8-year project!) LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it? Our finding: LLM representations have implicit symbolic structure! Link in thread ⬇️ 1/n

45 270 1K 210K 1K

View Details

Arash @incrementaliser

4 weeks ago

I was just thinking this *moving fast* that apparently everyone is doing is going to have consequences. Moving fast to where? for what goal? with what plan and based on what forecast?

Utah teapot 🫖 ➡️ Eleos ConCon and SF bay sep18-26 @SkyeSharkie

4 weeks ago

Okay, so since I got laid off, I can actually explain a huge problem I saw from the inside with regard to industry practices on training models. I won't say specifically where I worked, but I worked at an outsource training provider that was focused on RLVR training data for

61 166 2K 448K 1K

1 0 2 153 0

View Details

Arash @incrementaliser

a month ago

At this speed of model release, it makes sense to train a "pick the best LLM" classifier so we don't even have to switch models manually every time a new one is introduced. 🫩

0 0 1 41 0

View Details

David Chalmers @davidchalmers42

a month ago

an analysis of anthropic's "J-space": is it a global workspace and does it have the key features of access consciousness? (spoiler: not quite.) philpapers.org/rec/CHAITJ-2 on whether J-space is a global workspace: 1. The "workspace-like" properties of J-space are not distinctive properties of the global workspace theory. 2. The J-space is not a (causal, dedicated, privileged, unified) system. 3. The J-space supports only a limited form of broadcast. 4. The J-space lacks other key computational features of the global workspace including modules, recurrent processing, and others. on whether J-space has the key features of access consciousness (n.b. phenomenal consciousness mostly isn't the issue here): 5. Many contents of access consciousness are not verbalizable. 6. What matters for the J-space may not be effects on verbal outputs, but effects on outputs in general. 7. Many verbalizable representations are not access conscious. 8. Verbalizability is not the same as reportability. overall: the J-space is a powerful tool for interpretability, but its status as a key causal mechanism and as a mechanism of access consciousness remains to be demonstrated.

25 102 567 42K 338

View Details

Arash @incrementaliser

a month ago

@tallinzen @PNASNews (although it's computationally expensive, and not strictly incremental)

0 0 0 26 0

View Details

Arash @incrementaliser

a month ago

@tallinzen @PNASNews Very interesting work! I’m now thinking if bidirectional models adapted in a restart-incremental fashion (arxiv.org/abs/2402.13113) are better at predicting the human regressions and rereading compared to recurrent or autoregressive models 1/2

1 0 0 122 0

View Details

Travers @travers00

2 months ago

I think we are entering a period of the largest industrial demand for human originality in history

54 74 640 35K 109

View Details

Arash @incrementaliser

a month ago

@leerob One feature (more on the UX side) I'd like is a "note mode": when editing some text, I'd try to judge it first and leave comments on the side, and then come back to fix them. It would be great to have this commenting option to then have an agent to "resolve/elaborate".

0 0 1 184 0

View Details

Christopher Potts @ChrisGPotts

2 months ago

@incrementaliser @apaz_cli @coreauto @robert_csordas @chrmanning @sarahookr @huggingface Thanks for this! It's probably better in some sense to go deeper, up to a point. (At some point, everyone is revealed be "not very good at optimization"!) However, simply scaling by depth could be a serious misallocation of your resources.