Download twitter comments with media for a tweet.

What I want to do:

I use https://www.awesomescreenshot.com/ to create this without downloading.

I want to download three things from a tweet

1. The main post for which I’ll provide a URL. In this case, I want to download this particular tweet by Matthew Kobach. I do want to capture the media files and the main text for this post. (MAIN TEXT)
2. The next thing I need is the comments on this tweet. As you can see, the first comment is by Trenton who follows up with an image. This is called user-generated content and it’s very valuable to bottle up this content for future use. I really liked the examples most people put up, so I wanted to download them for my personal use. (IMAGE)
3. At the same time, I want the text that Trenton wrote too. (TEXT).

Caveats:

You can have multiple comments on each comment. I really didn’t need those. I was able to find a solution to exclude this though.

Things that did not work

1. I tried to use PhantomBuster but they didn’t have any predefined scripts. I cannot code, so I wasn’t able to create a custom script to scrape the tweet.
2. Parsehub may or may not do the job. I left it out because my Macbook Air sounds like a 747 jet taking off whenever I open Parsehub.
3. Saving the file as HTML or MHT won’t work. At least it didn’t work for me and only sourced 1/20th of the page at max.
4. I tried to find if other solutions existed online. Had someone already created a small tool to do this?

Things that worked — a little

I found this that did half the job.

1

Exporting the twitter comments from a post worked. This was the result that I got after I entered the url in exportcomments.com

Results

We get the URLS on the right that go back to the tweet’s comment and url.

But we don’t get the media. Many times, users go back and delete their post. Having the tweet url is a little unstable.

I tried searching for another tool that would bring me just the url and the media. But I wasn’t able to find it.

How I made it work — without coding

Here’s the hack.

I want to bulk paste all these urls into a tool.

But I can’t do so with the hyperlink. So here’s what you’ll do to convert it into normal links.

<wow, I got stuck here>

Tried everything including scripts to macros. Nothing really worked. Scripts work when the =hyperlink formula is used. This is the only way I could get the urls.

1. Copy the url from the previous excel sheet.
2. Paste it into a new sheet (SHEET 1)

Go to File>Publish to the web. Use these settings to publish.

copy the link, paste it somewhere temporarily.

4. Create a new sheet.

5. Go to cell E1 and paste this formula

=IMPORTXML(“URL”,”//a/@href”)

6. Copy the published url that you generated and replace the ‘URL’ in the formula with it.

7. Wait for a few seconds and it should populate.

That’s it.

Now, we need to find a tool that converts these individual urls to media and text formats.

AFAIK, the only solution is to use code. I will update it here.