The Wayback Machine - https://web.archive.org/web/20240930173348/https://www.geeksforgeeks.org/remove-duplicates-given-string-python/
Open In App

Remove All Duplicates from a Given String in Python

Last Updated : 07 May, 2024
Summarize
Comments
Improve
Suggest changes
Like Article
Like
Save
Share
Report
News Follow

We are given a string and we need to remove all duplicates from it. What will be the output if the order of character matters? Examples:

Input : geeksforgeeks 
Output : geksfor

This problem has an existing solution please refer to Remove all duplicates from a given string. 

Method 1: 

Python
from collections import OrderedDict 

# Function to remove all duplicates from string 
# and order does not matter 
def removeDupWithoutOrder(str): 

    # set() --> A Set is an unordered collection 
    #         data type that is iterable, mutable, 
    #         and has no duplicate elements. 
    # "".join() --> It joins two adjacent elements in 
    #             iterable with any symbol defined in 
    #             "" ( double quotes ) and returns a 
    #             single string 
    return "".join(set(str)) 

# Function to remove all duplicates from string 
# and keep the order of characters same 
def removeDupWithOrder(str): 
    return "".join(OrderedDict.fromkeys(str)) 

# Driver program 
if __name__ == "__main__": 
    str = "geeksforgeeks"
    print ("Without Order = ",removeDupWithoutOrder(str)) 
    print ("With Order = ",removeDupWithOrder(str)) 

Output
Without Order =  okergsf
With Order =  geksfor

Time complexity: O(n)
Auxiliary Space: O(n)

Method 2: 

Python
def removeDuplicate(str):
    s = set(str)
    s = "".join(s)
    print("Without Order:", s)
    
    t = ""
    for i in str:
        if i in t:
            pass
        else:
            t = t + i
            
    print("With Order:", t)

str = "geeksforgeeks"
removeDuplicate(str)

Output
Without Order: soerkgf
With Order: geksfor

Time complexity: O(n)
Auxiliary Space: O(n)

What do OrderedDict and fromkeys() do?

An OrderedDict is a dictionary that remembers the order of the keys that were inserted first. If a new entry overwrites an existing entry, the original insertion position is left unchanged.

For example, see below code snippet : 

Python
from collections import OrderedDict 

ordinary_dictionary = {} 
ordinary_dictionary['a'] = 1
ordinary_dictionary['b'] = 2
ordinary_dictionary['c'] = 3
ordinary_dictionary['d'] = 4
ordinary_dictionary['e'] = 5

# Output = {'a': 1, 'c': 3, 'b': 2, 'e': 5, 'd': 4} 
print (ordinary_dictionary)     

ordered_dictionary = OrderedDict() 
ordered_dictionary['a'] = 1
ordered_dictionary['b'] = 2
ordered_dictionary['c'] = 3
ordered_dictionary['d'] = 4
ordered_dictionary['e'] = 5

# Output = {'a':1,'b':2,'c':3,'d':4,'e':5} 
print (ordered_dictionary)     

Output
{'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5}
OrderedDict([('a', 1), ('b', 2), ('c', 3), ('d', 4), ('e', 5)])

Time complexity: O(n)
Auxiliary Space: O(1)

fromkeys() creates a new dictionary with keys from seq and values set to a value and returns a list of keys, fromkeys(seq[, value]) is the syntax for fromkeys() method. Parameters :

  • seq : This is the list of values that would be used for dictionary keys preparation.
  • value : This is optional, if provided then the value would be set to this value.

For example, see below code snippet : 

Python
from collections import OrderedDict
seq = ('name', 'age', 'gender')
dict = OrderedDict.fromkeys(seq)

# Output = {'age': None, 'name': None, 'gender': None}
print (str(dict)) 
dict = OrderedDict.fromkeys(seq, 10)

# Output = {'age': 10, 'name': 10, 'gender': 10}
print (str(dict))       

Output
OrderedDict([('name', None), ('age', None), ('gender', None)])
OrderedDict([('name', 10), ('age', 10), ('gender', 10)])

Time complexity: O(n)
Auxiliary Space: O(1)

Method 5: Using operator.countOf() Method

Python
import operator as op


def removeDuplicate(str):
    s = set(str)
    s = "".join(s)
    print("Without Order:", s)
    t = ""
    for i in str:
        if op.countOf(t, i) > 0:
            pass
        else:
            t = t+i
    print("With Order:", t)


str = "geeksforgeeks"
removeDuplicate(str)

Output
Without Order: rfogesk
With Order: geksfor

Time Complexity: O(N)
Auxiliary Space : O(N)



Similar Reads

Python | Remove all duplicates words from a given sentence
Given a sentence containing n words/strings. Remove all duplicates words/strings which are similar to each others. Examples: Input : Geeks for Geeks Output : Geeks for Input : Python is great and Java is also great Output : is also Java Python and great We can solve this problem quickly using python Counter() method. Approach is very simple. 1) Spl
7 min read
Remove all consecutive duplicates from the string
Given a string S, The task is to remove all the consecutive duplicate characters of the string and return the resultant string. Note: that this problem is different from Recursively remove all adjacent duplicates. Here we keep one character and remove all subsequent same characters. Examples: Input: S= "aaaaabbbbbb"Output: ab Input: S = "geeksforge
14 min read
Remove All Adjacent Duplicates in String II
Given a string s and an integer k, the task is to repeatedly delete k adjacent duplicates till no deletions are possible and then return the final string. On deletion of k adjacent duplicates, the left and right sides of the deleted substring is concatenated together. Examples: Input: s = "abcd", k = 2Output: "abcd"Explanation: There's nothing to d
3 min read
Javascript Program To Remove Duplicates From A Given String
Write a javascript program for a given string S which may contain lowercase and uppercase characters. The task is to remove all duplicate characters from the string and find the resultant string. Note: The order of remaining characters in the output should be the same as in the original string.Example: Input: Str = geeksforgeeksOutput: geksforExpla
3 min read
Python groupby method to remove all consecutive duplicates
Given a string S, remove all the consecutive duplicates. Examples: Input : aaaaabbbbbb Output : ab Input : geeksforgeeks Output : geksforgeks Input : aabccba Output : abcba We have existing solution for this problem please refer Remove all consecutive duplicates from the string link. We can solve this problem in python quickly using itertools.group
2 min read
Remove all occurrences of duplicates from a sorted Linked List
Given a sorted linked list, delete all nodes that have duplicate numbers (all occurrences), leaving only numbers that appear once in the original list. Examples: Input : 23->28->28->35->49->49->53->53 Output : 23->35 Input : 11->11->11->11->75->75 Output : empty List Note that this is different from Remove Dup
10 min read
Recursively remove all adjacent duplicates
Given a string S, the task is to remove all its adjacent duplicate characters recursively. Examples: Input: S = "geeksforgeek"Output: "gksforgk"Explanation: g(ee)ksforg(ee)k -> gksforgk Input: S = "abccbccba"Output: ""Explanation: ab(cc)b(cc)ba->abbba->a(bbb)a->aa->(aa)->"" (empty string) Table of Content [Naive Approach] Using Re
13 min read
Remove duplicates from a string in O(1) extra space
Given a string str of lowercase characters, the task is to remove duplicates and return a resultant string without modifying the order of characters in the original string. Examples: Input: str = "geeksforgeeks" Output: geksfor Input: str = "characters" Output: chartes Approach: The idea is to use bits of a counter variable to mark the presence of
13 min read
Remove duplicates from string keeping the order according to last occurrences
Given a string, remove duplicate characters from the string, retaining the last occurrence of the duplicate characters. Assume the characters are case-sensitive. Examples: Input : geeksforgeeks Output : forgeks Explanation : Please note that we keep only last occurrences of repeating characters in same order as they appear in input. If we see resul
8 min read
Remove three consecutive duplicates from string
Given a string, you have to remove the three consecutive duplicates from the string. If no three are consecutive then output the string as it is. Examples: Input : aabbbaccddddcOutput :ccdcInput :aabbaccddcOutput :aabbaccddcRecommended PracticeThree consecutive duplicatesTry It!Explanation : We insert the characters of string one by one to vector a
12 min read
Remove duplicates from a string
Given a string s which may contain lowercase and uppercase characters. The task is to remove all duplicate characters from the string and find the resultant string. Note: The order of remaining characters in the output should be the same as in the original string. Example: Input: s = geeksforgeeksOutput: geksforExplanation: After removing duplicate
10 min read
Count of ways to empty given String by recursively removing all adjacent duplicates
Given a string S, in one move it is allowed to remove two adjacent equal characters. After the removal, both endpoints of the removed characters are joined. Calculate the total number of ways to empty the string. Example: Input: S = aabccbOutput: 3Explanation:1. aabccb -> aabb -> aa 2. aabccb -> aabb -> bb3. aabccb -> bccb -> bbHe
10 min read
Print all distinct permutations of a given string with duplicates
Given a string that may contain duplicates, write a function to print all permutations of given string such that no permutation is repeated in output.Examples: Input: str[] = "AB" Output: AB BA Input: str[] = "AA" Output: AA Input: str[] = "ABC" Output: ABC ACB BAC BCA CBA CAB Input: str[] = "ABA" Output: ABA AAB BAA Input: str[] = "ABCA" Output: A
15 min read
Python Remove Duplicates from a List
The job is simple. We need to take a list, with duplicate elements in it and generate another list that only contains the element without the duplicates in them. Examples: Input : [2, 4, 10, 20, 5, 2, 20, 4] Output : [2, 4, 10, 20, 5] Input : [28, 42, 28, 16, 90, 42, 42, 28] Output : [28, 42, 16, 90] We can use not in on list to find out the duplic
3 min read
Python | Remove consecutive duplicates from list
In Python, we generally wish to remove the duplicate elements, but sometimes for several specific usecases, we require to have remove just the elements repeated in succession. This is a quite easy task and having a shorthand for it can be useful. Let's discuss certain ways in which this task can be performed. Method #1 : Using groupby() + list comp
4 min read
Python - Ways to remove duplicates from list
This article focuses on one of the operations of getting a unique list from a list that contains a possible duplicate. Removing duplicates from list operation has a large number of applications and hence, its knowledge is good to have in Python. Ways to Remove duplicates from the list:Below are the methods that we will cover in this article: Using
10 min read
Remove duplicates from a sorted linked list using recursion
Write a removeDuplicates() function which takes a list sorted in non-decreasing order and deletes any duplicate nodes from the list. The list should only be traversed once. For example if the linked list is 11->11->11->21->43->43->60 then removeDuplicates() should convert the list to 11->21->43->60. Algorithm: Traverse th
8 min read
Remove duplicates from unsorted array using Map data structure
Given an unsorted array of integers, print the array after removing the duplicate elements from it. We need to print distinct array elements according to their first occurrence. Examples: Input : arr[] = { 1, 2, 5, 1, 7, 2, 4, 2} Output : 1 2 5 7 4 Explanation : {1, 2} appear more than one time. Approach : Take a hash map, which will store all the
4 min read
Remove duplicates from unsorted array using Set data structure
Given an unsorted array of integers, print the array after removing the duplicate elements from it. We need to print distinct array elements according to their first occurrence. Examples: Input: arr[] = { 1, 2, 5, 1, 7, 2, 4, 2} Output: 1 2 5 7 4 Explanation: {1, 2} appear more than one time. Input: arr[] = { 3, 3, 4, 1, 1} Output: 3 4 1 Approach:
3 min read
Remove duplicates from a dataframe in PySpark
In this article, we are going to drop the duplicate data from dataframe using pyspark in Python Before starting we are going to create Dataframe for demonstration: C/C++ Code # importing module import pyspark # importing sparksession from pyspark.sql module from pyspark.sql import SparkSession # creating sparksession and giving an app name spark =
2 min read
Remove duplicates from a sorted doubly linked list
Given a sorted doubly linked list containing n nodes. The problem is removing duplicate nodes from the given list. Examples: Algorithm: removeDuplicates(head_ref, x) if head_ref == NULL return Initialize current = head_ref while current->next != NULL if current->data == current->next->data deleteNode(head_ref, current->next) else cur
12 min read
Remove duplicates from an array of small primes
Given an array of primes arr[] such that the range of primes is small. Remove duplicates from the array. Examples: Input: arr[] = {3, 5, 7, 2, 2, 5, 7, 7}Output: 2 3 5 7Explanation: There are no duplicates among {2, 3, 5, 7}. Input: arr[] = {3, 5, 7, 3, 3, 13, 5, 13, 29, 13}Output: 3 5 7 13 29Explanation: There are no duplicates among {3, 5, 7, 13,
15 min read
Remove duplicates from an unsorted doubly linked list
Given an unsorted doubly linked list containing n nodes, the task is to remove duplicate nodes while preserving the original order. Examples: Input: Doubly Linked List = 1 <-> 2 <-> 3 <-> 2 <-> 4Output: Doubly Linked List = 1 <-> 2 <-> 3 <-> 4 Input: Doubly Linked List = 8 <-> 4 <-> 4 <->
15 min read
Remove Duplicates from an Unsorted Linked List
Given an unsorted linked list containing n nodes, the task is to remove duplicate nodes while preserving the original order. Examples: Input: 12 -> 11 -> 12 -> 21 -> 41 -> 43 -> 21 Output: 12 -> 11 -> 21 -> 41 -> 43 Explanation: The second occurrence of 12 (the one after 11) and the second occurrence of 21 (the one at
14 min read
Remove duplicates from a sorted linked list
Given a linked list sorted in non-decreasing order. Return the list by deleting the duplicate nodes from the list. The returned list should also be in non-decreasing order. Example: Input : Linked List = 11->11->11->21->43->43->60Output : 11->21->43->60Explanation: Input : Linked List = 5->10->10->20Output : 5-
15+ min read
Remove duplicates from Sorted Array
Given a sorted array arr[] of size N, the task is to remove the duplicate elements from the array. We need keep order of the remaining distinct elements as it was in the original array. Examples: Input: arr[] = {2, 2, 2, 2, 2}Output: arr[] = {2}Explanation: All the elements are 2, So only keep one instance of 2. Input: arr[] = {1, 2, 2, 3, 4, 4, 4,
7 min read
Check if string is palindrome after removing all consecutive duplicates
Given a string str, the task is to remove all the consecutive duplicates from the string str and check if the final string is palindrome or not. Print "Yes" if it is a palindromic else print "No". Examples: Input: str = "abbcbbbaaa" Output: Yes Explanation: On removing all consecutive duplicates characters, the string becomes "abcba" which is a pal
7 min read
Minimum cost to remove all 1s from given Binary String as per given conditions
Given a binary sequence of 1's and 0's. Our task is to remove all 1s from the sequence in minimum cost by below operations. Remove an element from the left end (i.e., remove s[0]) which costs 1 coin.Remove an element from the right end (i.e., remove s[s.length - 1]) which costs 1 coin.Remove an element from anywhere in the sequence which costs 2 co
7 min read
Maximise distance by rearranging all duplicates at same distance in given Array
Given an array arr[] of N integers. Arrange the array in a way such that the minimum distance among all pairs of same elements is maximum among all possible arrangements. The task is to find this maximum value. Examples: Input: arr[] = {1, 1, 2, 3}Output: 3Explanation: All possible arrangements are: {1, 1, 2, 3}, {1, 1, 3, 2}, {1, 2, 1, 3}, {1, 2,
6 min read
Python | Consecutive duplicates all elements deletion in list
Sometimes, while working with Python list, a problem can occur to filter list to remove duplicates. The solution to this has been discussed before. But sometimes, we may have a problem in which we need to delete the duplicate and element itself if it occurs more than 1 in consecution. This type of problem can occur in day-day programming and other
4 min read
Article Tags :
Practice Tags :