As mentioned in the previous posts duplicate content or thin content (Content that adds very little value for the user) should be avoided on the website. Removing such contents helps you in enhancing the search rankings of your website.
Duplicated content may not have been created intentionally, so it is good if you cross check the URLs of your site that have been indexed in Google.
Please find below some examples of duplicated content that could have been created unintentionally.
Let’s now explore the ways to identify if there is duplicate content on your website.
1. Use Google search
If you have a site with less than 100 pages this simple solution should help.
This will list all the pages of your site that Google has indexed. Check the URLs and see if there is more than one URL listing the same content.
In case you find some content that is repeating you can deep dive by using additional search parameters
Site:YourSiteName.com duplicate content
2. Get Google search result in spreadsheet
In case you prefer working with a spreadsheet you can extract the search results on a Google spreadsheet. Copy this file from below URL to your drive and modify the input fields in it to get all your search results in a spreadsheet.
3. Record the URLs that are visited by users and bots
In case you are a large organisation and have thousands/ millions of pages indexed in Google the above options may not be able to provide you with a complete view. Most of the organisation’s record data of URLs visited. This could be separated based on real users and bots. You could use the URLs crawled by bots and find the URLs that are duplicates.
If you are a victim of a duplication because of the theme you can easily migrate to a theme that just lists the snippet of the text on the home page and provides the entire blog post content on the internal page.
I will write on the other solutions available to handle duplicate content in my next post. In case you have any questions on the current post please feel free to post your comments below.