• Visitors can check out the Forum FAQ by clicking this link. You have to register before you can post: click the REGISTER link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. View our Forum Privacy Policy.
  • Want to receive the latest contracting news and advice straight to your inbox? Sign up to the ContractorUK newsletter here. Every sign up will also be entered into a draw to WIN £100 Amazon vouchers!

Have you done Natural Language Understanding?

Collapse
X
  •  
  • Filter
  • Time
  • Show
Clear All
new posts

    Have you done Natural Language Understanding?

    I'm on a project where they're trying to automate reading incoming emails and directing them to work queues.

    They've decided they can scan the mails for certain words or strings. I said it's more difficult than that. You need to either parse the text to get some idea of sentence structure or use analytics that looks for millions of correlations involving frequency, other words near by etc etc.

    So far it's failing badly. I've said it will never work but I could be wrong of course.

    I just wondered whether you've seen this kind of thing and can pass on any pointers eg did you have limitations on mail format, mandatory title words etc. Did you succeed with string searching logic or did you have to use more sophisticated software?
    "Don't part with your illusions; when they are gone you may still exist, but you have ceased to live" Mark Twain

    #2
    Originally posted by Cirrus View Post
    I'm on a project where they're trying to automate reading incoming emails and directing them to work queues.

    They've decided they can scan the mails for certain words or strings. I said it's more difficult than that. You need to either parse the text to get some idea of sentence structure or use analytics that looks for millions of correlations involving frequency, other words near by etc etc.

    So far it's failing badly. I've said it will never work but I could be wrong of course.

    I just wondered whether you've seen this kind of thing and can pass on any pointers eg did you have limitations on mail format, mandatory title words etc. Did you succeed with string searching logic or did you have to use more sophisticated software?
    I tend to agree with this point, seems like it would be very hard to achieve.
    The Chunt of Chunts.

    Comment


      #3
      Originally posted by Cirrus View Post
      I'm on a project where they're trying to automate reading incoming emails and directing them to work queues.

      They've decided they can scan the mails for certain words or strings. I said it's more difficult than that. You need to either parse the text to get some idea of sentence structure or use analytics that looks for millions of correlations involving frequency, other words near by etc etc.

      So far it's failing badly. I've said it will never work but I could be wrong of course.

      I just wondered whether you've seen this kind of thing and can pass on any pointers eg did you have limitations on mail format, mandatory title words etc. Did you succeed with string searching logic or did you have to use more sophisticated software?
      Only place I've seen it work is when emails are generated from a web based contact form where subject lines and key words can be forced by making the user select from pre-populated drops downs for different issues. Even then you need a catch all option at the bottom for the stuff you haven't thought of, and if the categories aren't obvious most users will use that anyway because their issue is "special".
      "Being nice costs nothing and sometimes gets you extra bacon" - Pondlife.

      Comment


        #4
        I was able to do something similar with DTS on SQL Server 2000 years ago. No pressure
        The greatest trick the devil ever pulled was convincing the world that he didn't exist

        Comment


          #5
          Didn't Google just open source some kind of natural language parsing system?

          Though I wonder if the output of something like that is just as hard to deal with as the input. I remember being asked the same question probably 10-15 years ago and having the same response.

          http://www.wired.com/2016/05/google-...ural-language/
          Last edited by VectraMan; 4 August 2016, 09:42. Reason: Linky
          Will work inside IR35. Or for food.

          Comment


            #6
            Have a look at Azure NLP.

            https://azure.microsoft.com/en-gb/do...s-quick-start/

            For more in depth understanding, you are talking cutting edge AI.

            https://www.microsoft.com/cognitive-...mentation/home
            https://www.luis.ai/Help/
            Last edited by DimPrawn; 4 August 2016, 10:04.

            Comment


              #7
              Originally posted by Cirrus View Post
              ... did you have to use more sophisticated software?
              For it to work, you need to use fairly sophisticated machine learning algorithms requiring proper computer scientists.
              Down with racism. Long live miscegenation!

              Comment


                #8
                Originally posted by NotAllThere View Post
                For it to work, you need to use fairly sophisticated machine learning algorithms requiring proper computer scientists.
                Rubbish. It depends on how complex you want to be. If I could do something in 2004 on SQL Server 2000, why the hell do you need machine learning algorithms?
                The greatest trick the devil ever pulled was convincing the world that he didn't exist

                Comment


                  #9
                  Many thanks - I'll share some of your comments.

                  This reminds me of something that I experience more and more: people being in positions of power - ie can decide what should be done - when they actually don't know what they're talking about.

                  Well that's not the problem I suppose.

                  The problem is they never listen to me when I advise them on what they should do
                  "Don't part with your illusions; when they are gone you may still exist, but you have ceased to live" Mark Twain

                  Comment


                    #10
                    Originally posted by Cirrus View Post
                    Many thanks - I'll share some of your comments.

                    This reminds me of something that I experience more and more: people being in positions of power - ie can decide what should be done - when they actually don't know what they're talking about.

                    Well that's not the problem I suppose.

                    The problem is they never listen to me when I advise them on what they should do
                    Ultimately it depends how complex/complete you want it to be. You could go for anything from forwarding the messages to a SQL Server inbox and using its mail functionality to sift them (and produce stats as it goes) to dreaming of spending millions on a project that doesn't get past the cost benefits stage.
                    The greatest trick the devil ever pulled was convincing the world that he didn't exist

                    Comment

                    Working...
                    X