At this point it’s clear that the Internet has become at least as important to a pop song’s success as old-school terrestrial radio. When Billboard began including digital streams in its formula for determining placement on the Hot 100 in late 2012 it radically altered the chart’s makeup, and since then songs that haven’t been promoted by their labels as singles (including meme-connected songs like Baauer’s “Harlem Shake” and Ylvis’s “The Fox (What Does the Fox Say?)”) have charted almost entirely off of YouTube plays. Including digital streams in their mainstream charts went a long way towards bringing Billboard‘s rankings in line with how people are actually consuming pop music.

Now they’re attempting to take that idea even further with something called the Billboard Twitter Real-Time Charts, which is pretty much exactly what the name says: a ranking of what songs are making waves on Twitter, compiled and released “multiple times a week,” according to an official announcement. If acknowledging the importance of YouTube and Spotify helped to make the Hot 100 once again relevant, shouldn’t a chart based on one of the more crucial platforms for developing songs, artists, and entire genres of music be an even greater step forward?

Maybe, or maybe not. Twitter is a great platform for breaking new music. Word of mouth has always been a crucial component of breaking a new artist, and the one thing Twitter does best is amplifying word of mouth. And now, at what seems likely to be the peak of the platform’s power, a single tweet from an established star is enough to put a newcomer on the map.

But Twitter is also extremely noisy, and extracting useful data from it—especially if you’re attempting to produce something that’s supposed to be as authoritative as Billboard considers its charts to be—can be brutally difficult. The partnership’s official announcement says that the Billboard Twitter Real-Time Charts “will reflect the top tracks being discussed at the moment and over an extended period of time on Twitter, as well as surface the most talked about and shared songs by new and upcoming acts,” but there’s been no indication yet what methodology they’ll use to determine those criteria, and the methodology is what’s going to determine whether the BTRTC will succeed or fail.

Judging the popularity of music, or in fact pretty much any thing or concept, on Twitter requires a lot of filtering. Running a universal search on an act’s name and tallying up the number of returns you get is massively ineffective when you’re dealing with a platform that’s perfectly engineered for shit-talking, so judging their popularity, and not just the extent to which people are talking about them (which are two very different things), would require sorting out irrelevant mentions.

The BTRTC is also intrinsically set up to be gamed. There are a lot of acts with negligibly small but passionate fan bases who are eager to be deployed in social media campaigns. (Cf. the Lawrence Arms’ continued domination of the Reader‘s Best of Chicago polls despite being one of the city’s more marginally relevant groups.) And while Vulfpeck got a lot of Twitter traction out of their admittedly brilliant Spotify hack, does that actually mean anything?

If Twitter and Billboard manage to sort out the signal from the noise, their Real-Time Charts could be interesting, and at best a truer reflection of what we’re listening to than we have right now. If not it’ll just be a tally of who’s being talked about most on Twitter, which anyone who’s already on there can figure out without the BTRTC’s help.