609. Find Duplicate File in System

Given a list of directory info including directory path, and all the files with contents in this directory, you need to find out all the groups of duplicate files in the file system in terms of their paths.

A group of duplicate files consists of at least two files that have exactly the same content.

A single directory info string in the input list has the following format:

"root/d1/d2/.../dm f1.txt(f1_content) f2.txt(f2_content) ... fn.txt(fn_content)"

It means there are n files (f1.txt, f2.txt ... fn.txt with content f1_content, f2_content ... fn_content, respectively) in directory root/d1/d2/.../dm. Note that n >= 1 and m >= 0. If m = 0, it means the directory is just the root directory.

The output is a list of group of duplicate file paths. For each group, it contains all the file paths of the files that have the same content. A file path is a string that has the following format:

"directory_path/file_name.txt"

Example 1:

Input:
["root/a 1.txt(abcd) 2.txt(efgh)", "root/c 3.txt(abcd)", "root/c/d 4.txt(efgh)", "root 4.txt(efgh)"]
Output:
[["root/a/2.txt","root/c/d/4.txt","root/4.txt"],["root/a/1.txt","root/c/3.txt"]]

Note:

1. No order is required for the final output.
2. You may assume the directory name, file name and file content only has letters and digits, and the length of file content is in the range of [1,50].
3. The number of files given is in the range of [1,20000].
4. You may assume no files or directories share the same name in the same directory.
5. You may assume each given directory info represents a unique directory. Directory path and file info are separated by a single blank space.

Follow-up beyond contest:

1. Imagine you are given a real file system, how will you search files? DFS or BFS?
2. If the file content is very large (GB level), how will you modify your solution?
3. If you can only read the file by 1kb each time, how will you modify your solution?
4. What is the time complexity of your modified solution? What is the most time-consuming part and memory consuming part of it? How to optimize?
5. How to make sure the duplicated files you find are not false positive?

 Approach #1: C++.

class Solution {
public:
    vector<vector<string>> findDuplicate(vector<string>& paths) {
        int size = paths.size();
        unordered_map<string, vector<string>> mp;
        for (int i = 0; i < size; ++i) {
            int found = paths[i].find(' ');
            string str = paths[i].substr(0, found);
            while (found != string::npos) {
                int last = found;
                found = paths[i].find(' ', last+1);
                mp[str].push_back(paths[i].substr(last+1, found-last-1));
            }
        }

        unordered_map<string, vector<string>> temp;

        for (auto m : mp) {
            string base = m.first + "/";
            for (string s : m.second) {
                int fre = s.find('(');
                int las = s.find(')');
                string key = s.substr(fre+1, las-fre-1);
                string kid = s.substr(0, fre);
                temp[key].push_back(base+kid);
            }
        }

        vector<vector<string>> ans;

        for (auto it : temp) {
            vector<string> ant;
            for (string s : it.second) {
                ant.push_back(s);
            }
            if (ant.size() >1)
                ans.push_back(ant);
        }

        return ans;
    }
};

　　

Approach #2: Java.

class Solution {
    public List<List<String>> findDuplicate(String[] paths) {
        HashMap<String, List<String>> map = new HashMap<>();
        for (String path : paths) {
            String[] values = path.split(" ");
            for (int i = 1; i < values.length; ++i) {
                String[] name_cont = values[i].split("\\(");
                name_cont[1] = name_cont[1].replace(")", "");
                List<String> list = map.getOrDefault(name_cont[1], new ArrayList<String>());
                list.add(values[0] + "/" + name_cont[0]);
                map.put(name_cont[1], list);
            }
        }
        List<List<String>> res = new ArrayList<>();
        for (String key : map.keySet()) {
            if (map.get(key).size() > 1)
                res.add(map.get(key));
        }
        return res;
    }
}

　　

Apparoch #3: Python.

class Solution(object):
    def findDuplicate(self, paths):
        """
        :type paths: List[str]
        :rtype: List[List[str]]
        """
        M = collections.defaultdict(list)
        for line in paths:
            data = line.split()
            root = data[0]
            for file in data[1:]:
                name, _, content = file.partition('(')
                M[content[:-1]].append(root + '/' + name)
        return [x for x in M.values() if len(x) > 1]

　　

Analysis:

In this question our goal is to split and combine the string. If you are familiar with the operate it will easy to solve this problem.

C++ -----> string:assign

string (1)	string& assign (const string& str);
substring (2)	string& assign (const string& str, size_t subpos, size_t sublen);
c-string (3)	string& assign (const char* s);
buffer (4)	string& assign (const char* s, size_t n);
fill (5)	string& assign (size_t n, char c);
range (6)	template <class InputIterator> string& assign (InputIterator first, InputIterator last);

Assign content to string

Assigns a new value to the string, replacing its current contents.

(1) string

Copies str.

(2) substring

Copies the portion of str that begins at the character position subpos and spans sublen characters (or until the end of str, if either str is too short or if sublen is string::npos).

(3) c-string

Copies the null-terminated character sequence (C-string) pointed by s.

(4) buffer

Copies the first n characters from the array of characters pointed by s.

(5) fill

Replaces the current value by n consecutive copies of character c.

(6) range

Copies the sequence of characters in the range [first,last), in the same order.

(7) initializer list

Copies each of the characters in il, in the same order.

(8) move

Acquires the contents of str.
str is left in an unspecified but valid state.

// string::assign
#include <iostream>
#include <string>

int main ()
{
  std::string str;
  std::string base="The quick brown fox jumps over a lazy dog.";

  // used in the same order as described above:

  str.assign(base);
  std::cout << str << '\n';

  str.assign(base,10,9);
  std::cout << str << '\n';         // "brown fox"

  str.assign("pangrams are cool",7);
  std::cout << str << '\n';         // "pangram"

  str.assign("c-string");
  std::cout << str << '\n';         // "c-string"

  str.assign(10,'*');
  std::cout << str << '\n';         // "**********"

  str.assign<int>(10,0x2D);
  std::cout << str << '\n';         // "----------"

  str.assign(base.begin()+16,base.end()-12);
  std::cout << str << '\n';         // "fox jumps over"

  return 0;
}

　　

C++ -----> string:substr.

string substr (size_t pos = 0, size_t len = npos) const;

Generate substring

Returns a newly constructed string object with its value initialized to a copy of a substring of this object.

The substring is the portion of the object that starts at character position pos and spans len characters (or until the end of the string, whichever comes first).

Parameters

pos

Position of the first character to be copied as a substring.
If this is equal to the string length, the function returns an empty string.
If this is greater than the string length, it throws out_of_range.
Note: The first character is denoted by a value of 0 (not 1).

len

Number of characters to include in the substring (if the string is shorter, as many characters as possible are used).
A value of string::npos indicates all characters until the end of the string.

size_t is an unsigned integral type (the same as member type string::size_type).