[:en]The diversity of Atlanta’s 242 neighborhoods captivated me. Some neighborhoods are indistinguishable, while others possessed vivid personalities. There is Cabbagetown, an eclectic, hip community with historic shotgun-style homes. Walking from Cabbagetown through the Krog Street Tunnel, you will discover walls of evolving street art. Nearby Inman Park has its own character with Queen Anne, Colonial Revival, Jacobean Revival and Shingle houses surrounded by verdant nature. Drive further north and you will encounter Buckhead, with its massive mansions, mid-rise residences, skyscrapers, ritzy restaurants, and exceptional shopping.

Figure 1: The “Welcome to Inman Park” sign prominently displays the Inman Park Neighborhood Association’s registered trademark – a butterfly that symbolizes two faces, celebrating the past and the future of this neighborhood. Opened in 2014, the Krog Street Market, a hall featuring gourmet food booths and local crafts, represents the new development in Inman Park.

Between 2010 and 2015, the city of Atlanta gained more than 40,000 new residents. By 2040, metro Atlanta is predicted to have more than 8 million residents, an increase of the 5.7 million today. This fast-paced growth continuously alters neighborhoods that have become home to both transplants and locals. As one of these transplants, I have lived in Atlanta long enough to see development projects like the Atlantic Station, the Beltline, and Ponce City Market reshape sections of the city.  Sometimes I feel like a tourist in my own city as I stumble into new shops and restaurants. This varied and evolving vibe of Atlanta neighborhoods intrigued me and became the focal point of my master’s project at Georgia Tech. My project was inspired by the neighborhoods feature on Airbnb, where local editors highlight the personalities of neighborhoods in various major cities. I was curious to see if, through the use of social media data, one could discover neighborhood vibe profiles that would change alongside neighborhood transformations.

Figure 2: Some familiar sights in Buckhead include skyscrapers, shopping, and traffic.

Understanding Social Media Data

I chose Twitter as the initial step into my data science project. Twitter is a public platform for nonstop immediate expression of thoughts and sentiments through tweets that can also include geotags—critical for geography-based analysis. As appealing a source as Twitter is for data science, it also has inherent challenges.

• Tweets are naturally short. At just 140 characters, users can be resourceful, employing abbreviations and slang. Creating dictionaries for translating non-standard or locale specific usage is critical. In addition, document clustering algorithms work best with large documents containing several topics. (Note: Since the completion of my master’s project, Twitter has relaxed the 140 character limit.)
• Tweets are unstructured. There are few rules for how tweets are written. They may be any combination of words, short phrases, or complete sentences. In addition, they may have hashtags (like #riseup, used for Atlanta Falcons Football fans). These tags must be parsed into individual words for further analysis, and may require local knowledge to clarify meaning. Other challenges include emoticons, which can emphasize or reverse a sentiment. In addition, the tweets may have simple misspellings that must be corrected before analysis.
• Geotagged tweets may not represent the full population. Other research on Twitter shows that only a small percentage of users geotag their tweets, often because of privacy concerns. These willing users are sometimes travelers at major airports, parks, and public facilities. In other cases, they are non-personal accounts—such as marketers, bots, and spammers—that can deliberately weaken or reinforce the typecast of a neighborhood, as opposed to exposing novel encounters.
• Using only geotagged tweets limits the volume of data. I expected small neighborhoods would have fewer tweets than large neighborhoods. After cleaning/processing tweets, my most active Twitter neighborhoods averaged ~5,000 tweets per month whereas my least active neighborhoods averaged 300 tweets per month. The small data size added to the risk of not accurately representing the neighborhood population.
• Location may not match the topic or context of the tweet. While Twitter has the allure of immediacy, there is no guarantee that users will tweet at the moment of experience. To properly match a tweet to a neighborhood, context is needed to determine neighborhood relevancy; however, the short text limits the ability to derive context.

Collecting, Processing, and Analyzing Tweets

As a student, I had the freedom to explore in the face of challenges and determine if tweets could be used for neighborhood vibe research.

I used the Twitter Search API to collect geotagged English tweets within a 50-mile radius of Atlanta’s city center. Using neighborhood boundaries from mapping data and a tweet’s geotag, I associated each tweet to a neighborhood.

After data collection, I cleaned the data by replacing common misspellings, slang words, and abbreviations with English words from a custom dictionary of Atlanta specific slang, and abbreviations. I also parsed hashtags into separate words and removed all emoticons that could not be translated into relevant word meanings.

Because unsupervised document-clustering algorithms may not perform well with short tweets, I organized tweets into several large documents by neighborhoods, based on hashtags when available. I assumed tweets with the same hashtag would be related to each other. Tweets without hashtags were grouped together by Twitter account. In addition, I managed the impact of spam accounts by limiting the number of tweets contributed to a document by a single account. For example, there were non-personal accounts—for instance job recruiters or new nightclub marketers—that could have contributed more than 25% of the tweets for a neighborhood if left unchecked.

The tweet documents were input into Gensim, a free Python library for topic modeling/clustering. I then tweaked the parameters (number of topics, words per topics, and alpha and beta hyperparameters) to get a list of neighborhood topics. I manually classified the discovered topics into four vibe categories: points of interests, restaurants, perceptions/sentiments, and food/cuisine.

Putting the Vibe Topics to the Test

Understanding the many challenges Twitter-derived data presented, I had low initial expectations. However, I was pleasantly surprised to see distinct topics emerging per neighborhood. To determine if the topics were relevant, I ran a user study to assess two metrics: precision and recall. Precision measures the fraction of discovered topics that are relevant to a neighborhood; recall measures the fraction of relevant topics that were actually discovered.

I included the vibe topics for three neighborhoods with distinct vibes: Midtown, Virginia Highlands, and Cabbagetown. These three areas also had a varied volume of tweets, ranging from approximately 5,000 tweets per month in Midtown to Cabbagetown’s average of 300 tweets per month.

The user study was an anonymous online survey using SurveyMonkey. To recruit different population groups, I shared the survey on Facebook, Twitter, Yik Yak, and Atlanta subreddits, encouraging participants to also share with their friends and family. In formal terms, this is a snowball non-probability sampling method.

For each neighborhood, I included a Google classic map of its boundaries and asked potential to self-qualify based on recent knowledge (in the past six months) of the neighborhood. As people qualified, they proceeded to the topic questions. Each of the vibe topic categories  (the aforementioned points of interests, restaurants, perceptions/sentiments, and food/cuisine) had two questions:

• First, the survey presented a list of topics and asked participants to select topics that matched their understanding of that neighborhood (precision testing)
• The survey then asked participants to add topics that they believed were missing from that category (recall testing).

Precision

I calculated precision based on the number of topics that participants confirmed as matching their knowledge of that neighborhood. To eliminate outliers, a topic was included in the precision calculation only if at least 10% of the participants confirmed relevancy. Based on these criteria, I found that three out of four topics were relevant to a neighborhood. The food category performed worst, with only one out of two topics being relevant. This could be due to personal bias for certain cuisines.

Despite attempts to limit the effect of marketers, they still had an impact on precision.  For example, Club Kapture appeared as a top topic for Midtown, yet none of the survey participants found it relevant.

I also observed that participants did not always confirm a point of interest with low ratings as being relevant, even when the location was within a neighborhood’s boundaries.

The survey also showed that neighborhood boundaries are fuzzy. For example, Righteous Room and Majestic are in Poncey Highlands, which borders the Virginia Highlands. Both places appeared in the list of topics for Virginia Highlands. However, some participants adamantly opposed including them in Virginia Highlands, saying they belonged to Poncey Highlands. Others confirmed relevancy within the Virginia Highlands, perhaps feeling they were close enough to be included in that neighborhood.

This might also be an example of survey acquiescence bias—participants wanted to please the researcher by confirming their relevancy. Arguing for this explanation, there were other points of interests that participants confirmed to be associated with Virginia Highlands that were nowhere near its borders. For example, participants confirmed the relevancy of Smith’s Olde Bar to Virginia Highlands, yet the bar is in Piedmont Heights, separated from Virginia Highlands by a whole neighborhood.

Recall

I then calculated the number of topics that were suggested from participants via tweet analysis.  I compared the participants’ suggestions to the topics found from one month of tweets and found that recall was 20%. This means that the topic modeling only found 20% of the potential topics for that neighborhood. Because of the low recall rate, I expanded the topic modeling to use three months of tweets.

This low recall rate likely reflected how geotagged tweets do not represent the population and are therefore missing key topics for a neighborhood vibe. For example, Midtown is known for being an LGBT-friendly neighborhood; however, none of the topics that emerged from the data analysis were LGBT-related.

Participants also complained that favorite local hangout spots were missing. For example, Moe’s and Joe’s, a fixture in the Virginia Highlands for 67 years, did not appear in enough geotagged tweets to become a top topic.  This is likely caused by either geotagged tweets not representing the population or Twitter users not tweeting about their common daily experiences.

I also found that ambiguous proper names led to misclassification of favorite hot spots. For example, Home Grown is a popular restaurant in Cabbagetown, but I manually classified it as homegrown vegetables. Little’s Food Store is a neighborhood grocery store also in Cabbagetown; however, Little’s may have been misclassified as an adjective. Finally, Yeah! Burger, a popular burger joint in Midtown, may have been misclassified as an affirmation (Yeah!).

Cabbagetown is known for its street art. Also, displayed here is Little’s Food Store, established in 1929 in the heart of Cabbagetown.

Analyzing Temporal Vibe

For my final analysis, I compared topics from the same neighborhoods—from Spring 2015 to Spring 2016—hoping to find that some neighborhoods would show how the vibe was evolving.  I was disappointed to find no dramatic differences and few that could resulted from chance. For example, Spring 2016 introduced tweets about the Atlanta Beltline, an urban revitalization project to convert a railway corridor that encircles Atlanta to walking trails. Tweets could have increased because of new developments near the Atlanta Beltline. I also saw a rise in voting topics in 2016, however, 2016 was a Presidential election year. Mentions of exhibits at the High Museum surged in 2016, which may have reflected improved marketing or more interesting exhibit changes to the neighborhood vibe.

My project was a proof of concept to see if vibe analysis was possible with the use of geotagged tweets. Motivated by Airbnb’s curated neighborhood feature, in place of leveraging local subject matter experts, I explored how social media may describe the neighborhood vibe. Vibe can be derived from tweets on subjects like food, music, activity, art, and events.

Since social media data can have context, for example, location and time, it can reflect the up-to-the-minute vibe. The feel of a neighborhood can adjust based on time of day, day of the week, or season of the year. An obvious example is how a neighborhood can evolve from a business/corporate crowd to a hopping nightlife in a span of a day. These are nuances that can be overlooked when constructing a careful descriptive narrative for a neighborhood.

In addition to current vibe, social media may allow us to discover tipping points or emerging trends. Discovering these trends can steer important decisions, such as home-buying, site selections for restaurants or retail, and even city revitalization projects.

The vibe of a neighborhood is often dictated by the people congregating within them. My analysis could have included ascertaining categories of personalities for Twitter users and determining which neighborhoods attract those users.

As a proof of concept, I was able to see both the weaknesses and strengths of using social media data to understand neighborhood vibe.

 [:zh]某研究项目使用社交媒体数据来探究每个社区异彩纷呈的氛围。由于 Twitter 是一个通过推文（还可以附加地理标签）不断即时表达想法和情绪的公共平台，因此该项目选择它作为数据的基本来源。对相应主题的用户测试表明，采用包含本地名称和主题标签的详尽字典非常重要，而且需要具备一定深度和代表性的数据来源。

文章全文为英文版[:KO]한 연구 프로젝트는 각 이웃 지역의 다양한 분위기를 알아내기 위해 소셜 미디어 데이터를 사용했습니다. 트윗은 위치 정보 태그가 포함되어 있을 수도 있는 트윗을 통해 생각과 감정을 즉각적으로 표현할 수 있는 공개 플랫폼이기 때문에 데이터의 기반으로 선택되었습니다. 결과로 초래된 주제를 검증하는 사용자는 심층적이고 대표적인 데이터 소스의 필요성뿐만 아니라 지역 명칭과 해시 태그를 고려한 올바른 사전의 중요성을 보여줍니다.

전체 기사는 영어로만 제공됩니다.[:pt]Um projeto de pesquisa utilizou dados de mídias sociais para detectar as várias percepções do ambiente de cada área. O Twitter foi escolhido como base para os dados por ser uma plataforma pública para veiculação imediata e ininterrupta de ideias e opiniões por meio de tweets que também podem incluir marcações geográficas (geotags). Os tópicos resultantes do teste que foi feito com os usuários mostram a importância de contar com bons dicionários que levam em consideração nomes de locais e hashtags, bem como a necessidade de fontes de dados representativas e detalhadas.

O artigo completo está disponível somente em inglês.[:ja]あるリサーチプロジェクトでは、各地域の様々な空気 (感情、考え方、雰囲気など) を検知するためにソーシャルメディアのデータを用いた。ツィートを通じて常時考えや感情を即時に表現する、ジオタグを含む公共のプラットフォームであることから、ツイッターがデータの基盤として選ばれた。結果として得られたトピックをユーザーがテストしたところ、地域的な名称とハッシュタグを考慮に入れた優良な辞書が重要であるとともに、深く掘り下げた代表的なデータソースが必要であることがわかった。

原文は英語だけになります[:es]Un proyecto de investigación usó los datos de los medios sociales para detectar las diferentes sensaciones de cada barrio. Como Twitter es una plataforma pública para expresar de forma inmediata y continua pensamientos y sentimientos a través de los tuits que también pueden incluir geoetiquetas, se eligió Twitter como la base para los datos. Probar con usuarios sobre los resultados muestra la importancia de contar con buenos diccionarios que tengan en cuenta las etiquetas y los nombres locales, además de la necesidad de fuentes de datos detalladas y representativas.

La versión completa de este artículo está sólo disponible en inglés[:]

Topics: Storytelling, Innovation, Data
Published in: February, 2017 in [:en]Cloudworking and Big Data[:zh]云端工作和大数据 [:KO]클라우드 작업과 빅데이터 [:pt]Cloudworking e Big Data [:ja]クラウドワーキングとビッグデータ[:es]Trabajo en la nube y macrodatos[:]

Comments are closed.